Origin Docs
GuidesScraping

Troubleshoot Website Theme Generation

Fix sitemap, scraping, and Gateway API issues in the website theme generation workflow.

Sitemap issues

404: Check robots.txt for sitemap location. Add a User-Agent header:

headers = {"User-Agent": "Mozilla/5.0"}
response = requests.get(sitemap_url, headers=headers)

XML parse errors: Use root.findall(".//{*}loc"). Decompress .gz sitemaps first.

Scraping issues

Empty content: Site may use client-side rendering — try Playwright or Selenium.

403 Forbidden: Respect robots.txt; reduce request rate; add User-Agent.

Gateway API issues

401

client = OpenAI(
    base_url="https://api.gateway.orgn.com/v1",
    api_key="sk-ollm-YOUR_API_KEY"
)

Model not found

Use underscore IDs: near_glm_4_7 — not near/GLM-4.6.

Empty output

if response and response.choices:
    print(response.choices[0].message.content)

Token limits

Truncate input:

MAX_CHARS = 20000
combined_content = combined_content[:MAX_CHARS]

TEE attestation (optional)

TEE models (near_*) produce Scanner receipts. ZDR models do not. See Verification.

Checklist

  • Sitemap URL correct
  • Scraped content non-empty
  • Valid sk-ollm-* key
  • Underscore model ID
  • HTTP 2xx and valid choices

On this page