Open Source Python Search Engines

SearXNG is a free and open-source metasearch engine designed to aggregate results from multiple search engines while prioritizing user privacy and anonymity. Instead of maintaining its own index, it queries numerous external search providers and merges the results into a single interface, increasing coverage and diversity of information. One of its core principles is privacy, as it does not track users, store personal data, or create search profiles, making it a strong alternative to traditional search engines. . The system can be self-hosted, allowing individuals or organizations to run their own private search instance with full control over configuration and data handling. It supports extensive customization, including selecting which engines to query, filtering content, and adjusting ranking or display behavior. Developers can also integrate it into other tools through its API, enabling automation and advanced workflows.

Anna’s Archive is a large-scale open-source search engine and data aggregation platform designed to index and provide access to a vast collection of books, academic papers, comics, magazines, and other digital texts through a unified interface. The project includes all the infrastructure required to run a full instance locally or in production, combining web servers, databases, and search indexing systems into a scalable architecture. It relies heavily on technologies such as Elasticsearch for search functionality and MariaDB for structured data storage, enabling fast and efficient querying across massive datasets. The system is designed with redundancy and replication in mind, allowing distributed deployments and mirrored environments to handle high traffic and large data volumes. It also includes tooling for importing datasets, managing metadata, and maintaining structured archives using custom formats.

Searx is a free and open source internet metasearch engine that respects your privacy. It aggregates results from over 70 search services, providing accurate, relevant results with absolutely no user tracking or profiling going on. It’s like having your very own private search engine. It has secure, encrypted connections and does not use cookies by default, giving you the assurance that your searches are 100% private and secure. Searx can also be used over Tor, providing you online anonymity. Searx can be easily integrated with any search engine of your choice. Start searching with absolute privacy and peace of mind with Searx!

SpiderFoot is an open source intelligence automation tool. Its goal is to automate the process of gathering intelligence about a given target, which may be an IP address, domain name, hostname or network subnet. SpiderFoot can be used offensively, i.e. as part of a black-box penetration test to gather information about the target or defensively to identify what information your organisation is freely providing for attackers to use against you.

Torrent Search is a cross-platform application, allowing to search for torrent files on different websites. Supported websites are integrated through plugins, which allows to easily extend the number of websites supported.

This is an ***old archive*** of tools developed for facilitating the use of Creative Commons licenses and metadata. --- For the most up to date representation of any of the projects listed here, please see: http://creativecommons.org/project/Developer.

Zope is an open source application server specializing in content management, intranets, and custom web applications. Zope is written in Python and has a large, global community of developers and companies.

googler is a power tool to Google (Web & News) and Google Site Search from the command-line. It shows the title, URL and abstract for each result, which can be directly opened in a browser from the terminal. Results are fetched in pages (with page navigation). Supports sequential searches in a single googler instance. googler was initially written to cater to headless servers without X. You can integrate it with a text-based browser. However, it has grown into a very handy and flexible utility that delivers much more. For example, fetch any number of results or start anywhere, limit search by any duration, define aliases to google search any number of websites, switch domains easily... all of this in a very clean interface without ads or stray URLs. The shell completion scripts make sure you don't need to remember any options. googler isn't affiliated to Google in any way. Demo: https://asciinema.org/a/85019

Bookmark-Manager is an advanced bookmark management utility for Windows supporting importing/exporting and merging of Internet Explorer favorites, Opera hotlists, Mozilla, Netscape, and Firefox bookmarks, XBEL, and HTML lists.

ddgr is a cmdline utility to search DuckDuckGo from the terminal. While googler is highly popular among cmdline users, in many forums the need of a similar utility for privacy-aware DuckDuckGo came up. DuckDuckGo Bangs are super-cool too! So here's ddgr for you! Unlike the web interface, you can specify the number of search results you would like to see per page. It's more convenient than skimming through 30-odd search results per page. The default interface is carefully designed to use minimum space without sacrificing readability. ddgr isn't affiliated to DuckDuckGo in any way. Demo: https://asciinema.org/a/151849

HarvestMan is a fully functional, multithreaded webcrawler cum offline-browser. It is highly customizable and supports as much as 55 plus options for controlling and customizing offline browsing. It is written entirely in the Python programming language.

Lupy is a full text indexer for Python. It is a port of Jakarta Lucene 1.2 to Python. Specifically, it reads and writes indexes in Lucene binary format. Like Lucene, it is sophisticated and scalable.

Infinispan is an open source, Java based data grid platform. ***IMPORTANT*** Starting with Infinispan 5.0.0.FINAL, Infinispan releases are no longer hosted in Sourceforge. They can now be located in www.jboss.org/infinispan/downloads

That project aims at providing a clean API, and the corresponding C++ implementation, for parsing travel-focused requests (e.g., "washington dc beijing monday r/t +aa -ua 1 week 2 adults 1 dog").

A threaded Web graph (Power law random graph) generator written in Python. It can generate a synthetic Web graph of about one million nodes in a few minutes on a desktop machine. It implements a threaded variant of the RMAT algorithm.

Pagecast makes it easy to submit lists of URLs to Internet search engines. Advanced features include: ability to check the URL's for certain problematic conditions. Designed to be simple to use, but effective.

Pyndex is a simple and fast full-text indexer and Bayesian classifier implemented in Python. It uses Metakit as its storage back-end. It works well for quickly adding search to an application, and is also well suited to in-memory indexing and search.

a small collection of python 3000 scripts/modules used to automate searching craigslist.org cities and categories for interesting stuff; these scripts currently use html screen scraping, since craigslist currently has no api

A simple app that builds a library of your music, then fetches information from metal-archives, discogs, etc. and shows you what you're still missing (e.g. new releases). Moved to http://github.com/KenjiTakahashi/gayeogi

HyperSQL is like a doxygen plus javadoc for SQL, hypermapping SQL views, packages, procedures, and functions to HTML source code listings and showing all code locations where these are used.

SE Auditor is a program for analyzing web pages for search engines. SE Auditor is application that you can use to view statistical data about your website, in order to improve its position within the Web search results. SE Auditor is addressed to SEO professionals, website designers, developers, website testers and owners. SE Auditor enables you to check meta description, keywords, sitemap, the number of links and keyword consistency, the text/HTML ratio and many more ranking / usability / social factors. Unlike other programs of this type SE Auditor is: - free - fast, do not waste your time (the audit process goes smooth as butter) - simple to understand interface (you can see at a glance) - shows the data important in optimizing your site - based on the collected data allows to generate a PDF report with hints - is accurate (for example in counting words, characters, calculating text/html ratio..)

Squishdot is a web-based news publishing and discussion product for Zope. It is written in Python with the ability to script in DTML and runs on Linux, *BSD, Unix and Windows 95/98/NT/200.

ViewVC is a web-based version control repository browser, currently supporting CVS and Subversion repositories. Project development is now hosted at https://github.com/viewvc/viewvc/.

Software, information, data sets and documentation for the Web as Corpus community.

Fetches topics with new posts from ZetaBoards forums and does something with the URLs, like opening them in a browser. Configurations can be stored and manipulated for quicker fetching. Development, translations, bug reports, etc. are handled at Launchpad: https://launchpad.net/zb-fetcher SourceForge is used to host released files.