How a directory gets built, in order

Pick the niche

A directory works when the market is fragmented, no single site owns it, and there's a question buyers ask that nobody answers well. For off-road, that question is rig compatibility. The existing apps are navigation — they show you where a trail goes, not whether your specific vehicle survives it. So Trail Scout is the discovery and trip-planning layer, not another map.

The test: name the one filter your competitor doesn't have. If you can't, keep looking.

Find a public data source

Never scrape a competitor's listings to seed a competing directory. The good news is that for most physical-world niches a government agency already publishes the underlying dataset for free. Here it's the Forest Service Motor Vehicle Use Maps, served as ArcGIS REST endpoints — thousands of legally designated routes with geometry and vehicle-class rules attached.

Two more sources queued: BLM travel routes, because the Moab classics sit on BLM land rather than Forest Service, and Recreation.gov for campgrounds near trailheads.

Write the puller

One Python file. It looks up the map layer by name instead of by ID, so it keeps working when the agency renumbers their services. It walks each state by bounding box, pages through the features, and normalizes every row into a clean record — including a derived flag for whether full-size rigs are legally allowed. Raw responses get saved untouched alongside the normalized output, so nothing is lost in translation.

Prove the pipe live

Offline self-test first, then one deliberately tiny live pull — a single state, capped at 500 features. It confirmed the layer lookup resolves, the paging works against production servers, and every field the normalizer reads actually exists on live records. What it did not confirm is whether the values are any good. That's what the sanity report in step 07 is for.

Gotcha: the normalized files are rewritten from scratch on every run, so pull all the states you want in one command or you'll only keep the last batch.

Write the standing brief

A single markdown file in the project root that Claude Code reads at the start of every session. It lists what already works and must not be rewritten, the data-model traps, the current task list, and the house rules. Without it every new session starts by guessing — and the first thing it guesses wrong is that a working script needs refactoring.

The trap it exists to prevent: map rows are segments, not trails. One named trail is many rows. Miss that and every count on the site is wrong.

Load it into the database

Schema first, then a loader that upserts the normalized records in batches, keyed on ID so re-running it is always safe. The connection string lives in an environment file and never in the code. Raw segments land in their own table, untouched — every later table is derived from it, so a bad transform is always reversible.

Group segments into trails

The step that turns a pile of government records into a directory. Segments get grouped by route number, name, and forest into one row per real trail, each with a URL slug, total mileage, and a rolled-up access flag. Then a sanity report prints: how many segments, how many trails, what percentage allows full-size rigs, and the twenty longest routes. That report is the first honest look at whether the dataset is any good.

Enrich past what the government gives you

Public data gives you legality and geometry. It does not give you difficulty ratings, obstacle descriptions, photos, or recent conditions — and those are the things people actually search for. This is the crawl layer, and it's a one-time bulk ingest rather than an ongoing feed, which is why an open-source crawler beats a credit-metered service here. The paid tool earns its keep as the fallback for pages that block everything else.

Order matters: crawl only after the trails table exists. Enrichment needs something to attach to, or you're collecting orphaned text.

Generate the pages

One page per trail, generated from the database rather than written by hand, plus per-state and near-me pages built from the same rows. This is where the earlier discipline pays off: because trails were grouped properly, every page has a real name, real mileage, and a stable URL instead of a route number nobody searches for.

Make it findable

Segmented sitemaps so a few thousand pages actually get crawled, IndexNow for fast submission, Search Console to watch what gets indexed versus ignored, and a plain-text file describing the site for AI crawlers. Every trail page also carries a line telling people to verify against the current official map before they go — the printed map is the legal document, not this site.

Build the part nobody can copy

Anyone can pull the same public dataset — the seed is not the moat. The moat is the rig compatibility engine: matching a specific vehicle, on specific tires, with or without lockers, against a specific trail. That data is manual and community-contributed, it compounds slowly, and it's the reason someone comes back rather than bookmarking a map.