Jotform AI Agent not crawling all of 24 sub pages?

  • richeaton
    Asked on March 20, 2025 at 2:21 PM

    Using a URL to a page to train a Jotform AI Agent. It stops crawling after a few sub-pages. I deleted and started over. Again, it crawled a few different pages, stopped on a different page. It sometimes crawls 3 pages, sometimes crawls more pages. it is lostpetsofomahaarea.org. it has 24 sub-pages all with the information that my agent would need

    Jotform AI Agent not crawling all of 24 sub pages? Image 1 Screenshot 20

    Page URL: https://lostpetsofomahaarea.org/
  • Camille_E Jotform Support
    Replied on March 20, 2025 at 7:19 PM

    Hi Richard,

    Thanks for reaching out to Jotform Support. When I cloned and tested your form, everything worked the way it was supposed to. Take a look at the screenshot below to see my results:

    Jotform AI Agent not crawling all of 24 sub pages? Image 1 Screenshot 30
    Keep in mind that once you crawl a link, you can only crawl 10 links at a time. To crawl all 24 pages, you need to re-crawl the link 3 times, so the other links left out will be crawled to. Check out the sample screencast:

    Jotform AI Agent not crawling all of 24 sub pages? Image 2 Screenshot 41 Give it a try and let us know how it goes.