Open Source Python Internet Software - Page 43

SICKnotes (Site Inventory, Configuration, and Knowledge) is a permissions-based Web tool that assists with inventory, system changes, contact info, and knowledge for anything from a small startup to a data center. SICKnotes requires Python and MySQL.

Text based utility for SNMP scripting (coded in Python using PySNMP). Usefull to batch configure SNMP capable devices or batch read televalues. Python 2.4 and PySNMP libraries necessary.

SOAPy is a SOAP-1.1 library for Python which uses WSDL and SDL documents to discover SOAP-based service APIs. It also includes an XML Schema parser which can parse a subset of the XML Schema standard.

kcm_sshd is a python-based KControl module intended for configuring OpenSSHD servers. It supports many common options and integrates nicely into System Settings and other KDE configuration tools.

A simple python based tool for toggling the state of SSH Server in Debian systems featuring a small GTK2-GUI.

Code for reference implementations of identity brokers and simple single sign-on (SSO) mechanisms that utilize XDI and link contracts to manage the dataweb.

In supermodernistic times data has to be cultivated not merely stored. Proper cultivation involves p2p exchange, arbitrary linking, modular referential reusage, lowest-denominator publishing, collaborative authoring. Stefanix is another try to achive this

A cross-platform server installer tool.

The SaWALi Web Application Library is a Python application that aims to provide a reasonably complete set of components for operating a multi-purpose website. Taking advantage of the Pylons Framework, SaWALi is fully-customisable and inherently-extensible. All of SaWALi's administrative and public interfaces can be modified to suit a website's userbase— from its document editors and server error pages down to its public-facing pages and site maps. Being a Python module, SaWALi can also be used to power several websites operating in the same host. There is no need to have multiple instances of SaWALi, one for each domain or sub-domain even if it requires its own look-and-feel.

Named after a well know product from Microsoft, SandStorm is a framework for creating modular middle-end web products.

Sarfe stands for "Send And Receive Files Easily". It aims to provide an easy and efficient interface for file transfers.

Satrap is an online address management system. Users can change their own records via a web-based interface, but there are also special administration groups.

The OO component based XML processing framework.The goal is to make the XML developer's life easier.It's going ot achieve that by providing reusable components for all stages of XML processsing with easy language and application binding.

SchemaDoc is a XML-based markup language for documenting XML schemas. The work products include both the vocabulary and a set of tools for combining it with the schema source (e.g. a DTD) to produce documentation in HTML, XML DocBook, LaTeX, etc.

Extracting content from websites and local documents using LLM. ScrapeGraphAI is a web scraping python library that uses LLM and direct graph logic to create scraping pipelines for websites and local documents (XML, HTML, JSON, Markdown, etc.). Just say which information you want to extract and the library will do it for you.

Web spider and SERP scrapper

Scrapling is an adaptive web scraping framework designed to handle everything from a single HTTP request to large-scale, concurrent crawls. Built for modern websites, it intelligently adapts to structural changes by automatically relocating elements when page layouts update. The framework includes advanced fetchers capable of bypassing anti-bot protections such as Cloudflare Turnstile using stealth and browser automation techniques. Its powerful spider system supports multi-session crawling, pause and resume functionality, and real-time streaming of scraped data. Scrapling combines high performance, memory efficiency, and extensive async support to deliver blazing-fast scraping workflows. With a developer-friendly API, CLI tools, MCP server integration for AI-assisted extraction, and Docker support, it offers a complete solution for modern web scrapers.

You can start multiple spider instances that share a single redis queue. Best suitable for broad multi-domain crawls. Scraped items gets pushed into a redis queued meaning that you can start as many as needed post-processing processes sharing the items queue. Scheduler + Duplication Filter, Item Pipeline, Base Spiders. Default requests serializer is pickle, but it can be changed to any module with loads and dumps functions. Note that pickle is not compatible between python versions. Version 0.3 changed the requests serialization from marshal to cPickle, therefore persisted requests using version 0.2 will not able to work on 0.3. The class scrapy_redis.spiders.RedisSpider enables a spider to read the urls from redis. The urls in the redis queue will be processed one after another, if the first request yields more requests, the spider will process those requests before fetching another url from redis.

Scrapyd can manage multiple projects and each project can have multiple versions uploaded, but only the latest one will be used for launching new spiders. A common (and useful) convention to use for the version name is the revision number of the version control tool you’re using to track your Scrapy project code. For example: r23. The versions are not compared alphabetically but using a smarter algorithm (the same packaging uses) so r10 compares greater to r9, for example. Scrapyd is an application (typically run as a daemon) that listens to requests for spiders to run and spawns a process for each one. Scrapyd also runs multiple processes in parallel, allocating them in a fixed number of slots given by the max_proc and max_proc_per_cpu options, starting as many processes as possible to handle the load.

Web app for Scrapyd cluster management, with support for Scrapy log analysis & visualization. Make sure that Scrapyd has been installed and started on all of your hosts. Start ScrapydWeb via command scrapydweb. (a config file would be generated for customizing settings on the first startup.) Add your Scrapyd servers, both formats of string and tuple are supported, you can attach basic auth for accessing the Scrapyd server, as well as a string for grouping or labeling. You can select any number of Scrapyd servers by grouping and filtering, and then invoke the HTTP JSON API of Scrapyd on the cluster with just a few clicks.

Scratchy is an Apache log file parser and HTML report generator written in Python.

A Trac plugin to allow pasting screenshots or other images captured or copied in the clipboard directly as attachements to tickets, Wiki pages, etc., without the need to first saving as images and then uploading them. Once the plugin is installed in Trac, you can easily attach a screenshot or any image you have in the clipboard to a Ticket or Wiki page, with one click.

ScrollServer is a Python based HTTP server that provides an html interface to the ScrollKeeper documentation database. It allows you to browse all documentation that is registered in your ScrollKeeper database using any web browser.

Scweet is a Python-based Twitter/X scraping library and CLI designed to collect tweets, profile timelines, followers, following lists, and user profile data without requiring the official Twitter/X API or a developer account. Instead of depending on deprecated unauthenticated scraping methods, it works by using X’s web GraphQL API together with authenticated browser cookies, which gives it a more current and practical approach for data extraction. The project supports a broad set of collection patterns, including searches by keyword, hashtag, user, date range, engagement thresholds, language, and location, making it useful for research, monitoring, and data gathering workflows. It is built for both local use and higher-volume runs, with support for proxies, dedicated accounts, and multi-account cookie handling to improve reliability at scale. Scweet also includes asynchronous method variants, a command-line interface, automatic credential persistence in a local database, etc.

A web based application that can search newsgroup names very easy. Use can search use Regular Expression for names and search by popularity.