Search Results for "web crawler source code"
Sort By:
Showing 113 open source projects for "web crawler source code"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Build Securely on AWS with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
1
RobotsTxt
The repository contains Google's robots.txt parser
This is a high-performance, production-tested library for parsing and evaluating robots.txt rules against crawler user agents. It implements the core semantics of the Robots Exclusion Protocol: user-agent sections, Allow/Disallow directives, wildcard handling, and precedence rules. The code is optimized for speed and low memory so large crawls can evaluate millions of URLs quickly. It also focuses on correctness—edge cases like overlapping patterns and longest-match resolution are handled... -
2
iCn3D
Web-based protein structure viewer and analysis tool
"I see in 3D" (iCn3D) Structure Viewer is not only a web-based 3D viewer, but also a structure analysis tool interactively or in the batch mode using NodeJS scripts based on the npm package icn3d. iCn3D synchronizes the display of 3D structure, 2D interaction, and 1D sequences and annotations. Users' custom displays can be saved in a short URL or a PNG image. The complete package of iCn3D including Three.js and jQuery is in the directory "dist" after you get the source code with the "Code" button. ... -
3
OWASP Amass
In-depth attack surface mapping and asset discovery
The OWASP Amass Project has developed a tool to help information security professionals perform network mapping of attack surfaces and perform external asset discovery using open source information gathering and active reconnaissance techniques. The Open Web Application Security Project (OWASP) is a nonprofit foundation that works to improve the security of software. All of our projects ,tools, documents, forums, and chapters are free and open to anyone interested in improving application... -
4
STranslate
A ready-to-go translation ocr tool developed with WPF/WPF
STranslate is a lightweight, open-source machine translation front end that lets users translate text between languages using a variety of supported back-end translation engines or APIs, offering a simple GUI for quick translation tasks without needing to write code or use complex web UIs. The application is designed to be small, cross-platform, and flexible, giving users the ability to type or paste text and receive instant translations while optionally selecting the desired language pairs or switching between multiple service providers. ... -
Enterprise-grade ITSM, for every businessFreshservice is an intuitive, AI-powered platform that helps IT, operations, and business teams deliver exceptional service without the usual complexity. Automate repetitive tasks, resolve issues faster, and provide seamless support across the organization. From managing incidents and assets to driving smarter decisions, Freshservice makes it easy to stay efficient and scale with confidence.
-
5
PySchool
Installable / Portable Python Distribution for Everyone.
PySchool is a free and open-source Python distribution intended primarily for students who learn Python and data analysis, but it can also used by scientists, engineering, and data scientists. It includes more than 150 Python packages (full edition) including numpy, pandas, scipy, sympy, keras, scikit-learn, matplotlib, seaborn, beautifulsoup4... -
6
mySQM+ DIY SQM WEATHER STATION
ESP32 mySQM+ Sky-Meter/Weather Station with loads of features
mySQM+ Sky Quality Meter/Weather Station (c) R Brown, 2020-2025, All rights reserved. Permission is granted for personal and Academic use only. Code or portions of code may not be copied or used without appropriate credit given to author(s). EXPANSION BOARDS - You can expand this project using the two types of project expansion boards at https://sourceforge.net/projects/myesp-boards/ -
7
myFP2ESP8266 Focus Controller
This is a port of the myFP2ESP32 Wifi Focus Controller to the ESP8266
A ESP8266 WiFi focuser. DRV8825, ULN2003, L298N, L293DMINI , L9110S, Support for Webserver, ALPACA, and TCP/IP. (c) Robert Brown 2014-2025, Holger M, 2019-2021. All rights reserved. Code or portions of code may not be copied or used without appropriate credit given to author(s). -
8
myESP32S3-N16R8 Focuser
ESP32S3_DEVKITC_1 N16R8 Wifi focuser
An ESP32S3 WiFi Focuser (ALPACA-ASCOM-LINUX-INDI-TCPIP-WINDOWS) (c) Robert Brown 2014-2026, Holger M, 2019-2021. All rights reserved. Code or portions of code may not be copied or used without credit given to authors. Read Part1 pdf to determine the board you will need, then download those board files. Downloading all files will cause confusion and hinder your project progress. Both PCB and DIY are supported. Spreadsheet to calculate what stepper motor to... -
9
SPPAS
SPPAS - the automatic annotation and analyses of speech
SPPAS is a scientific computer software package written and maintained by Brigitte Bigi of the Laboratoire Parole et Langage, in Aix-en-Provence, France. Available for free, with open source code, there is simply no other package for linguists to simple use in the automatic annotations of speech, the analyses of any kind of annotated data and the conversion of annotated files. SPPAS is able to produce automatically speech annotations from a recorded speech sound and its orthographic... -
Forever Free Full-Stack Observability | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
10
bbbike
An information system for cyclists in Berlin and Brandenburg
A bike information system for cyclists in Berlin and Brandenburg (Germany) with a map view and a routing engine. There's a graphical interface (based on Perl/Tk) as well as an Web interface. Source code may be found at Github: https://github.com/eserte/bbbike/ -
11
...You can also connect directly with the companies supporting the development. Source code available on github https://github.com/geonetwork/
-
12
myFP2ESP32-Dev WiFi ASCOM Focuser
ASCOM-INDI ESP32-Dev Wifi focuser
An ESP32-Dev WiFi focuser. DIY or PCB designs. (c) Robert Brown 2014-2025, Holger M, 2019-2021. All rights reserved. Code or portions of code may not be copied or used without appropriate credit given to authors. Spreadsheet to calculate what stepper motor to use. https://sourceforge.net/projects/arduinoascomfocuserpro2diy/files/Documentation/Nema-Stepper-Motors.xlsx/download Spreadsheets to calculate... -
13
CleverPsych Practice Management
FREE software to manage your psychology or allied health solo practice
CleverPsych is FREE software for psychologists and other allied health professionals who run their own practices. We have been using CleverPsych in our own practice in Australia since 2007. The system enables you to record your clients, appointments, services, organisations and referring doctors. The aim is to make operations like generating standard letters, claiming from the government and insurance companies, and reporting income, much easier. Look in the Files tab for... -
14
GeoTools, the Java GIS toolkit
Toolkit for working with and mapping geospatial data
GeoTools is an open source (LGPL) Java code library which provides standards compliant methods for the manipulation of geospatial data. GeoTools is an Open Source Geospatial Foundation project. The GeoTools library data structures are based on Open Geospatial Consortium (OGC) specifications. -
15
Astrolog
Astrology calculation, charting, and analysis
Astrolog is astrology software featuring many types of computation, display, graphics, comparison, and analysis. It supports multiple environments, such as MS Windows and Unix X Windows, with complete C++ source code available. For more information see the Web site: http://www.astrolog.org/astrolog.htm -
16
PlateEditor
PlateEditor, a free web application to work with multi-well plates
...It doesn't require any installation. - PlateEditor is available at this address: https://plateeditor.sourceforge.io - The source code is also available on GitHub, visit: https://github.com/vindelorme/PlateEditor More information about the source code and the API are available there (the wiki is still under construction, thanks for your patience!) For a list of recent updates, visit: https://sourceforge.net/p/plateeditor/wiki/Updates/ Hoping PlateEditor will make your life easier in the lab! ... -
17
OpenClinic GA
Open Source Integrated Hospital Information Management System
OpenClinic GA is an open source integrated hospital information management system covering management of administrative, financial, clinical, lab, x-ray, pharmacy, meals distribution and other data. Extensive statistical and reporting capabilities. OpenClinic GA and OpenClinic GMAO are owned by Frank Verbeke, MD, PhD at Post-Factum BV (https://www.post-factum.be) -
18
DeepSee
Visualize deep ocean biogeochemical sediment samples in 2D and 3D!
DeepSee is an interactive workspace for deep ocean scientists to upload sediment core data and map images and see their sampling history displayed across multiple connected views at once! 🦑🦀🐚 🚀 For a live demo, visit: https://www.its.caltech.edu/~datavis/deepsee/ 🌱 To get started, visit our Wiki: https://sourceforge.net/p/deepsee/wiki/Home/ 🧑💻 To modify DeepSee for your own project, visit our GitHub repository: https://github.com/orphanlab/DeepSee/ --- Created by Adam Coscia,... -
19
Traccar (GPS tracking system)
Open source system for GPS tracking devices
...Application includes embedded web server to track devices on map. Official web page: https://www.traccar.org/ Demo: https://www.traccar.org/demo-server/ Source code repository: https://github.com/tananaev/traccar Issues tracker: https://github.com/tananaev/traccar/issues -
20
SGraph
Create charts by retrieving data from Stata on-the-fly
SGraph is a web application that interacts with Stata for creating graphs. It is a demo application that demostrates how to exploit SWire for creating web applications that communicate with Stata. SGraph is an open source project and you can download the code from the project web site web site or try the app on-line here: http://sgraph-with-swire.sourceforge.net/ The source project is hosted in GitHub: https://github.com/lomagno/sgraph -
21
ESP32 Dew Controller
ESP32 4-channel dew controller.
This is an enhanced 4-Channel automated Dew Controller based on an ESP32 WiFi chip on a PCB and enclosed in a project hobby box. (c) Copyright Robert Brown 2020-2024. All Rights Reserved.Permission is granted for personal and Academic use only. Code or portions of code may not be copied or used without appropriate credit given to author. EXPANSION BOARDS - You can expand this project using the two types of project expansion boards at https://sourceforge.net/projects/myesp-boards/ -
22
This project uses PHP Code to programing mathematics and theoritical physics it begins with finding the roots of nonlinear equation and Lagrange interpolation we will develop it step by step you can also download it from http://bsw.benhascience.com
-
23
aseryla
Aseryla code repositories
This project describes a model of how the semantic human memory represents the information relevant to the objects of the world in text format. It provides a system and a GUI application capable of extracting and managing concepts and relations from English texts. https://aseryla2.sourceforge.io/ -
24
openPOWERLINK
An Open Source POWERLINK Stack
openPOWERLINK is an Open Source Industrial Ethernet stack for communication in hard real-time. It can be operated as both, Managing Node (master) and Controlled Node (slave). With version 2.0, the source code has been cleanly split into an application-oriented user library and a time-critical stack driver. The latter may be moved to a dedicated communication processor or into a kernel module to deliver enhanced performance while still keeping the API in user space. ... -
25
queXC
Web based system for cleaning and classifying open text fields
An Open Source, web based data cleaning and coding system. queXC takes a data file (such as questionnaire data) and cleans the text input fields by spacing and spell checking them. Operators then code text fields using new/existing coding schemes.
