WebExtract to retrieve news articles from the online web sites, you can either use predefined url list as argument, or you can let it to iterate over pages (automated pagination).
Data stored in a proprietary format, serialized Article classes and compressed.

Example create serialized list of common.Article class by pagination
(see getSerializedList.bat)

Example create .idx and .gz files used by WebExtractServer
(see getCasByPaging.bat)

Example create .idx and .gz files used by WebExtractServer input from .url file
(see getByPagingFromURLFile.bat)
using urls file example.url

Features

  • simple usage
  • portability (java)
  • designed to be used with WebExtractServer
  • fetch from urls list or generate pagination
  • support wait between retrievals
  • handle network errors with its simple protocol
  • support regex pattern to indicate stop fetching
  • support skip previously fetched urls

Project Activity

See All Activity >

Follow WebExtractLte

WebExtractLte Web Site

Other Useful Business Software
Gemini 3 and 200+ AI Models on One Platform Icon
Gemini 3 and 200+ AI Models on One Platform

Access Google's best plus Claude, Llama, and Gemma. Fine-tune and deploy from one console.

Build generative AI apps with Vertex AI. Switch between models without switching platforms.
Start Free
Rate This Project
Login To Rate This Project

User Reviews

Be the first to post a review of WebExtractLte!

Additional Project Details

Registered

2019-04-29
Report inappropriate content