Scrape with Webᵀ Lite

Scrape on your machine using your own OpenAI API Key

Ensure you have installed Web Transpose Lite:

pip install webtranspose[lite]

Ensure you have your OpenAI API Key in your PATH.

Extract information from websites using a schema in one line of code.

import webtranpose as webt

schema = {
  "Merchant Name": "string",
  "Title of Product": "string",
  "Product Photo URL": "string",
}

scraper = webt.Scraper(
  schema, 
)

scraper.run("https://www.example.com")

Webᵀ Lite Scrape Parameters

schema · dict · required

The schema for the data to be scraped.

verbose · bool · default: false

Extra logging to help debugging.

scraper · webtranspose.OpenAIScraper

The OpenAI scraper to be used. (Optional)