Separate crawling from indexing
A crawler uses robots.txt to decide which URLs it may request. A search engine can learn that a URL exists without fetching its full content. That is why blocking a path is not the same as guaranteeing it will disappear from results.
Protect private material properly
Do not put confidential documents on a public URL and rely on a robots rule to protect them. Use authentication and access controls. For public pages that should not be indexed, review the appropriate noindex mechanism and ensure the crawler can actually observe it; a blocked request may prevent that observation.
Check the deployed file
Inspect the live robots.txt, including the hostname, path, status and text content. A successful HTTP response that contains an HTML error page is not a useful robots file. Review crawler-specific groups and sitemap links with a person who understands the desired policy. The lab reports whether it observed a plain-text file; it does not prove every rule permits or prevents access.