Scweet Files

Scweet v4 is a major rewrite focused on a cleaner surface and a more reliable core. Tweet search scraping is now API-only (Twitter/X web GraphQL), and Scweet manages local state (accounts, leases/cooldowns, resume checkpoints, manifest cache) in a SQLite DB.

Highlights

• API-only tweet search scraping (GraphQL SearchTimeline) using curl_cffi (sync + async).
• DB-first account provisioning into SQLite from:
• accounts.txt
• cookies.json and browser-exported Netscape cookies.txt
• .env via env_path
• cookies= payload (cookie dict/list, Cookie header string, auth_token string, JSON string, or file path)
• Unified config via ScweetConfig + Scweet.from_sources(...) (with overrides=... for quick patches).
• Account leasing + limits:
• Per-account proxy support (stored in DB; set via files or ScweetDB)
• Daily request/tweet caps + min delay between requests
• bootstrap_strategy for token vs nodriver cookie bootstrap (auto|token_only|nodriver_only|none)
• Output + resume
• Output formats: csv|json|both|none
• Resume modes: legacy_csv, db_cursor, hybrid_safe
• Optional resume dedupe: output.dedupe_on_resume_by_tweet_id
• Manifest support
• Default manifest shipped with the package
• Optional remote refresh on init (update_manifest=True)
• Maintenance helpers
• ScweetDB helpers (inspect accounts, reset cooldowns/leases, set per-account proxy, opt-in duplicate collapse by auth_token)

Breaking changes (vs v3)

• scrape() / ascrape() now returns list[dict] of raw GraphQL tweet objects (full fidelity).
• CSV output is now a curated, stable “important fields” schema derived from GraphQL.
• Use JSON output (or the returned list) if you need every field.

v3 Compatibility (v4.x)

• Legacy import still works: from Scweet.scweet import Scweet (deprecated; planned removal in v5.0).
• Legacy constructor args and scrape/ascrape signatures preserved.
• Legacy CSV naming and legacy resume semantics preserved when using the legacy facade.

Removed / Cleanups

• Old browser scraping engine codepaths. nodriver remains only for internal login/cookie bootstrap.
• Removed requests fallback for API HTTP (v4 uses curl_cffi).

Known limitations

• Profile/followers/following endpoints are stubbed or incomplete in v4.x.
• Twitter/X behavior can change at any time; account quality and request patterns impact reliability.

Recommended

• For production runs + scaling: use the hosted Apify Actor (free plan/trial available): https://apify.com/altimis/scweet?fpr=a40q9&fp_sid=jeb97
• Library docs: see DOCUMENTATION.md