DownloadSwirl queries any number of data sources with APIs and uses spaCy and NLTK to re-rank the unified results without extracting and indexing anything! Includes zero-code configs for Apache Solr, ChatGPT, Elastic Search, OpenSearch, PostgreSQL, Google BigQuery, RequestsGet, Google PSE, NLResearch.com, Miro & more! SWIRL adapts and distributes queries to anything with a search API - search engines, databases, noSQL engines, cloud/SaaS services etc - and uses AI (Large Language Models) to re-rank the unified results without extracting and indexing anything. It's intended for use by developers and data scientists who want to solve multi-silo search problems from enterprise search to new monitoring & alerting solutions that push information to users continuously. Built on the Python/Django/RabbitMQ stack, SWIRL includes connectors to Apache Solr, ChatGPT, Elastic, OpenSearch | PostgreSQL, Google BigQuery plus generic HTTP/GET/JSON with configurations for premium services.
Features
- Ranked results will appear in just a few seconds
- WIRL includes 3 Google Programmable Search Engines (PSEs)
- Complete with shared credentials, to get you up and running with right away
- SearchProvider configurations for all included Connectors. They can be organized with the active, default and tags properties
- Synchronous or asynchronous search federation via APIs
- Optional subscribe feature to continuously monitor any search for new results
- Pipelining of Processor stages for real-time adaptation and transformation of queries, responses and results
- Results stored in SQLite3 or PostgreSQL for post-processing, consumption and/or analytics
- Duplicate detection on field or by configurable Cosine Similarity threshold
- Matching on word stems and handling of stopword via NLTK
Project Samples
