Search Results for "web crawler bot"
Sort By:
Showing 33 open source projects for "web crawler bot"
View related business solutions-
Get full visibility and control over your tasks and projects with Wrike.Wrike offers world-class features that empower cross-functional, distributed, or growing teams take their projects from the initial request stage all the way to tracking work progress and reporting results.
-
Streamline Hiring with Skill AssessmentsCanditech offers innovative, cheat-proof skill assessments and job simulations to transform your hiring process. From technical skills to soft skills, we help you assess candidates on actual job performance. With over 500 customizable tests and powerful video interview features, you can evaluate real-world capabilities, streamline your hiring, and reduce biases. Whether you’re hiring for remote roles, mass hiring, or looking to expand your diversity pool, Canditech’s data-driven platform ensures the right candidates are chosen for the job every time.
-
1
Pholcus
Distributed high-concurrency crawler software written in pure golang
Pholcus is a high-concurrency crawler software written in pure Go language that supports distributed, only used for programming learning and research. It supports three operating modes of stand-alone, server and client, and has three operating interfaces, Web, GUI, and command line; simple and flexible rules, concurrent batch tasks, and rich output methods (mysql/mongodb/kafka/csv/excel, etc.); In addition, it also supports horizontal and vertical grabbing modes, and a series of advanced functions such as simulated login and task suspension and cancellation. ... -
2
SafeLine
Serve as a reverse proxy to protect your web services from attacks
SafeLine is a self-hosted WAF(Web Application Firewall) to protect your web apps from attacks and exploits. A web application firewall helps protect web apps by filtering and monitoring HTTP traffic between a web application and the Internet. It typically protects web apps from attacks such as SQL injection, XSS, code injection, os command injection, CRLF injection, LDAP injection, XPath injection, RCE, XXE, SSRF, path traversal, backdoor, brute force, HTTP-flood, bot abuse, among others. ... -
3
3X-UI
Xray panel supporting multi-protocol multi-user expire day
3X‑UI is an enhanced fork of the X‑UI project, serving as a multifunctional web panel to manage VPN services. It supports multi‑protocol configurations (like VMess, VLESS, Trojan, Shadowsocks, WireGuard), offers traffic and user limits, SSL management, multilingual support, and integrates Telegram notifications for system events. Advanced, open-source web-based control panel designed for managing Xray-core server. It offers a user-friendly interface for configuring and monitoring various VPN... -
4
Coraza
OWASP Coraza WAF is a golang modsecurity compatible firewall library
...CRS protects from many common attack categories including: SQL Injection (SQLi), Cross Site Scripting (XSS), PHP & Java Code Injection, HTTPoxy, Shellshock, Scripting/Scanner/Bot Detection & Metadata & Error Leakages. Coraza is a library at its core, with many integrations to deploy on-premise Web Application Firewall instances. -
Endpoint Protection Software for Businesses | HYPERSECUREThe HYPERSECURE endpoint protection platform is a comprehensive suite of products and services enhanced by European third-party solutions. It ensures our customers’ IT security, regulatory compliance, and digital sovereignty.
-
5
Kong
The Cloud-Native API Gateway
Kong is a next generation cloud-native API platform for multi-cloud and hybrid organizations. When building for the web, mobile, or Internet of Things, you’ll need a common functionality to run your software, and Kong is that solution. Kong acts as a gateway, connecting microservices requests and APIs natively while also providing load balancing, logging, monitoring, authentication, rate-limiting, and so much more through plugins. Kong is highly extensible as well as platform agnostic,... -
6
WFDownloader App
Free batch downloader for image, wallpaper, video, audio, document,
Use as an image gallery, wallpaper, audio/music, video, document, and other media bulk downloader from supported websites. Also use to download sequential website urls that have a certain pattern (e.g. image01.png to image100.png). Also use app's built-in site crawler for advanced link search or extraction. There is also special support for forum media and open directory downloading. It's a programmable downloader and also works with password protected sites. Say goodbye to downloading one... -
7
SPM - Monitoring system
Monitoring Tool for your IT Environment
...With SPM Monitoring Systems, you can monitor your network, servers, applications, and services with ease, and receive timely alerts when issues arise. Host Availability Monitoring. Agent-Based Monitoring: CPU,RAM,Storage,Network,EventLogs etc. SNMP Monitoring. Web Site Monitoring. Notification System: E-Mail, SMS, Push, Telegram Bot (also control via telegram bot) and Sound Notifications. Logging. Web Console. Control by Telegram Bot. Agents for Windows and Linux. SPM Monitoring Systems is a reliable and efficient solution for monitoring your IT environment. With its comprehensive features and flexible notification system, you can stay on top of your infrastructure's health and performance. -
8
AlertsOnWings
Pinger alert
AlertsOnWings is web-based interface for viewing a `heartbeat` of network by pinging hosts. No extra files, only DB. All reports generated "on-the-fly". And we got personal cabinet for each user. -
9
ostRAT
OpenSourceTelegramRAT - Remote PC access via Telegram Bot.
ostRAT is free and open source. GPLv3 Сomputer remote control software. Works via telegram bot. A lot of functions, for example: - Screenshot: sends a screenshot - Off: turns off the computer - Url: opens entered link - Write: sends your text to the computer - Move: changes mouse location with x and y - and more! WARNING: Using the bot is recommended only on your device. Failure to comply with the recommendation may result in criminal liability. -
PeerGFS PEER Software - File Sharing and CollaborationPeerGFS is a software-only solution developed to solve file management/file replication challenges in multi-site, multi-platform, and hybrid multi-cloud environments.
-
10
PHP mini vulnerability suite
Multiple server/webapp vulnerability scanner
github: https://github.com/samedog/phpmvs -
11
taskt
Free and open-source robotic process automation (rpa)
taskt (formerly sharpRPA) is the first truly free, easy-to-use, and open-source process automation client built on the .NET Framework in C#. taskt allows you to build and design process automation without needing to write application code. taskt allows you to automate the boring stuff and create efficienies by giving you the power to craft a digital workforce that executes and performs rule-based automation. No API? No Problem! Included is a "what you see is what you get" bot designer with... -
12
magnetW
Magnet link aggregation search
magnetW is based on the rule principle of magnetX , the search results of each magnetic station are uniformly formatted. There is no group in this project, only Github for code hosting and related technical exchanges, and other addresses may be risky, please distinguish carefully. This project is open source and free. There are no collection channels of any kind, such as donations, and no advertising of any kind. If you encounter anything similar to the above situation, please don't believe... -
13
OpenSearchServer Search Engine
An open source search engine with RESTFul API and crawlers
OpenSearchServer is a powerful, enterprise-class, search engine program. Using the web user interface, the crawlers (web, file, database, etc.) and the client libraries (REST/API , Ruby, Rails, Node.js, PHP, Perl) you will be able to integrate quickly and easily advanced full-text search capabilities in your application: Full-text with basic semantic, join queries, boolean queries, facet and filter, document (PDF, Office, etc.) indexation, web scrapping,etc. OpenSearchServer runs on... -
14
pyspider
A powerful Spider(Web Crawler) system in Python
pyspider is a powerful Spider(Web Crawler) system in Python. Components are connected by message queue. Every component, including message queue, is running in their own process/thread, and replaceable. That means, when process is slow, you can have many instances of processor and make full use of multiple CPUs, or deploy to multiple machines. This architecture makes pyspider really fast. benchmarking. -
15
diskover
File system crawler and disk space usage software
diskover is a file system crawler and disk space usage software that uses Elasticsearch to index your file metadata. diskover crawls and indexes your files on a local computer or remote storage server over network mounts. diskover helps manage your storage by identifying old and unused files and give better insights into data change "hotfiles", file duplication "dupes" and wasted space. It is designed to help deal with managing large amounts of data growth and provide detailed storage... -
16
autoreplybot
an 'auto-reply' bot for your mail server
autoreplybot is designed for mail server administrators wishing to add an easy-to-use, easy-to-implement and uninvasive "autoreply" service for their users. All you need is a PHP environement and a little postgresql database. The concept is fairly simple: * You configure your mail server to send a copy of every mail that it processes to a specific mailbox * You set up autoreplybot to poll this mailbox, either via POP3 or IMAP * Users log on the autoreplybot web panel using... -
17
DotNetWikiBot Framework
Make robots for MediaWiki-powered sites!
The DotNetWikiBot Framework was developed so that it can offer a helping hand with many complicated and routine tasks of wiki site development and maintenance. DotNetWikiBot Framework is a cross-platform full-featured client API, that allows you to build programs and web robots easily to manage information on MediaWiki-powered sites. DotNetWikiBot Framework can also be used for learning C# and .NET. -
18
Addons for IOSEC - DoS HTTP Security
IOSec Addons are enhancements for web security and crawler detection
...Added Facebook Bot Support for "Facebot/1.0". Add this code to your website to prevent unauthorized stealing of your valuable content & block malicious bots from crawling your site. -
19
GHZ Tools v0.6 Build 9645 Release Data (02/09/2014) 7zPass: MHg2NzY4N0E3NDZGNkY2QzczMzAzNj== (base64/hex) Properties: 1)- Brute Forcer: WordPress Joomla 4images osCommerce Drupal, Razor Ftp cPanel Whmcs DirectAdmin Authentication Bypass SSH Authentication vBulletin Kleeja OpenCart WordPress Xmlrpc 2)- Remote Exploits: JCE Webdav 3)- SQL Injector: Auto SQL Injection 4)- Hash Cracker: MD2 MD4 MD5 SHA1 MD5(MD5(PASS)) SHA1(SHA1(PASS)) 5)- URL Fuzzer: URL Fuzzer 6)- Web Scanner: RFI/LFI URL Scanner Web Extractor Open Port Scanner URL Crawler SQLi Scanner
-
20
Python Crawler Library
Python Web Crawler Library
A simple library for crawling the web. This library will give you the ability to create macros for crawling web site and preforming simple actions like preforming "log in" and other simple actions in web sites. -
21
The archive-crawler project is building Heritrix: a flexible, extensible, robust, and scalable web crawler capable of fetching, archiving, and analyzing the full diversity and breadth of internet-accesible content.
-
22
RiverGlass EssentialScanner is an open source web and file system crawler which indexes the text content of discovered files so they can be retrieved and analyzed. It provides simple scanner capabilities as part of larger enterprise search solutions.
-
23
SteelBot is an open source ICQ/Jabber bot written in PHP. It is fully modular and easy using, and can use files or MySQL engines as database driver.
-
24
Ex-Crawler is divided into 3 subprojects (Crawler Daemon, distributed gui Client, (web) search engine) which together provide a flexible and powerful search engine supporting distributed computing. More informations: http://ex-crawler.sourceforge.net
-
25
Agent based Regional Crawler strategy implementation - gathers users' common needs and interests in a certain domain. It crawls based on these interests, instead of crawling the web without any predefined order.
