Search Results for "java crawler"
Sort By:
Showing 9 open source projects for "java crawler"
View related business solutions-
Build with generative AI, deploy apps fast, and analyze data in seconds—all with Google-grade security.Google Cloud is a cloud-based service that allows you to create anything from simple websites to complex applications for businesses of all sizes.
-
Professional Email Hosting for Small Business | GreatmailDependable cloud based email hosting with spam filtering, antivirus protection, generous storage and webmail. Compatible with Outlook and all other POP3/IMAP clients. High volume SMTP service for responsible senders. Outbound relay service for transactional messages, email marketing campaigns, newsletters and other applications. Dedicated email servers, clustering and multiple IP load balancing for high volume senders. Fixed monthly cost with unlimited sending and reputation monitoring. Greatmail is an email service provider (ESP) specializing in business class email hosting, SMTP hosting and email servers. For ISPs, application programmers and cloud developers, we also provide custom solutions including dedicated IP servers and process specific, load balanced configurations with multiple servers.
-
1
WebMagic
A scalable web crawler framework for Java
WebMagic is a scalable crawler framework. It covers the whole lifecycle of crawler, downloading, url management, content extraction and persistent. It can simplify the development of a specific crawler. WebMagic is a simple but scalable crawler framework. You can develop a crawler easily based on it. WebMagic has a simple core with high flexibility, a simple API for html extracting. It also provides annotation with POJO to customize a crawler, and no configuration is needed. Some other... -
2
Heritrix
Internet Archive's open-source, web-scale, web crawler project
Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project. Heritrix (sometimes spelled heretrix, or misspelled or missaid as heratrix/heritix/heretix/heratix) is an archaic word for heiress (woman who inherits). Since our crawler seeks to collect and preserve the digital artifacts of our culture for the benefit of future researchers and generations, this name seemed apt. Heritrix is designed to respect the robots.txt exclusion directives† and... -
3
File System Crawler for Elasticsearch
Elasticsearch File System Crawler (FS Crawler)
This crawler helps to index binary documents such as PDF, Open Office, MS Office. Local file system (or a mounted drive) crawling and indexing new files, updating existing ones, and removing old ones. Remote file system over SSH/FTP crawling. REST interface to let you “upload” your binary documents to elastic search. -
4
A Java Web crawler: multi-threaded, scalable, with high performance, extensible and polite. It can be used to crawl and index any web or enterprise domain and is configurable through a XML configuration file.
-
Track time for payroll, billing and productivityBecause time is money, and we understand how challenging it can be to keep track of employee hours. The constant reminder to log timesheets so your business can increase billables, run an accurate payroll and remove the guesswork from project estimates – we get it.
-
5
Web-as-corpus tools in Java. * Simple Crawler (and also integration with Nutch and Heritrix) * HTML cleaner to remove boiler plate code * Language recognition * Corpus builder
-
6
LogCrawler is an ANT task for automatic testing of web applications. Using a HTTP crawler it visits all pages of a website and checks the server logfiles for errors. Use it as a "smoketest" with your CI system like CruiseControl.
-
7
JCrawler is a perfect cralwing/load-testing tool which is cookie-enabled and follows human crawling pattern (hit/second).
-
8
Content Engineering Tools including an XSLT based site rendering system, XSLT Documentation Generator, and Swing based Site Crawler. The tools may be downloaded and used seperately since there are no dependancies between them.
-
9
Luanium
A Lua-based crawling scripting language and leveraging selenium
...I would put commands in a file or DB to use selenium to interpret the HTML and Javascript. The best would be to have a complete language with conditionals and looping. I'm a java developper and I needed that the crawler to run in a Spring-Boot application. So I decided to use a Lua interpreter in Java to build a crawling tool based on Selenium. The trick here is to add the crawling commands into the Lua interpreter. -
InEight is a leader in construction project controls softwareMinimize risks, gain operational efficiency, control project costs, and make confident, informed decisions. InEight software has your back during every stage of construction, from accurate pre-planning to predictable execution and completion. When project teams collaborate effectively, every decision is backed by precise, authoritative insights.
- Previous
- You're on page 1
- Next
