An asynchronous website crawler! It isn’t very useful though, I don’t think Google engineers will be interested even if it makes lovely fart sounds, which it doesn’t. I wanted to write an interesting piece of code so I did partially. Which I agree, is a pain, but then it offers a more granular level of flexibility.As I mentioned in my previous post about Java 11 HTTP client, API comes with asynchronous method for making requests. You can wrap those in threads - so your java program has multiple threads of execution.īut then you run into the problem of synchronization. NIO will let you use sockets to connect to the web server - but then you have to manually create your own GET requests and parse the incoming HTTP headers/data.Īnother option is to use the classes - and you can find many tutorials for those online and on stackoverflow. Alex Martelli's answer talks about Java's NIO, which is a good answer if you are interested in implementing the HTTP protocol in your own code. I myself am not aware of an asynchronous HTTP client for Java. For example, Java does make some things more tedious - but usually that is to give the user more options in how, for example, an HTTP connection is executed, whereas actionscript might abstract details or possible errors away for ease of use. So, it is probably worth considering that actionscript and Java don't server the same niche. Be sure to read up on MultiThreadedHttpConnectionManager, it's what handles the connection pool, and it's not shown in the most basic example. Update: Here's the link to Apache HTTP Client. (Note that Apache HTTP Client does its own thread pooling, and the default configuration limits you to 20 threads max, and only two to each web server.) I think you'll like coupled with Apache HTTP Client, though it sounds like you'll need to do some customization for progress indication. It took just an hour or two to get it working reasonably well. The tester has a ThreadPoolExecutor that has n threads, and the Runnable tasks I feed it each fetch a page using Apache HTTP Client. Last month I wrote a load tester for a web server. I vividly remember writing a fetch thread pool for a web browser written in Java back in 1999, and it was a bear to get right. The Pool's afterExecute() method is called when each Runnable task completes. The ThreadPoolExecutor will queue up the Runnable tasks and feed them to available Threads in parallel. You then feed the pool any number of tasks to complete. You can set up a ThreadPoolExecutor that manages, say, four Threads. If you haven't looked at it already, check out the Java 5 - it makes multi-threaded apps much easier to develop. Private void httpStatusHandler(Event event) URLLoader loader = (URLLoader)event.target Private void completeHandler(Event event) fire it off - this is asynchronous so we handle URLRequest request = new URLRequest(url) Loader.addEventListener(IOErrorEvent.IO_ERROR, ioErrorHandler) Loader.addEventListener(HTTPStatusEvent.HTTP_STATUS, httpStatusHandler) Loader.addEventListener(Event.Complete, completeHandler) If you are not familiar with URLLoader in AS3, its so super easy and looks something like this: private void getURL(String url) ![]() What I am looking for is something easy to use from a developer point of view - something similar to URLLoader in AS3 - where you simply create a URLRequest - attach a bunch of event handlers to handle the completion, errors, progress, etc, and call a method to fire it off. Seeing as one doesn't seem to already exist, what is the best approach? Creating my own threads using a blocking type lib like httpclient or the built-in java http stuff, or should I use the newer non-blocking io java stuff - it seems particularly complex for something which should be simple. ![]() A primary example is a simple solution for asynchronous http requests. As a relative newbie in the Java world, I am finding many things frustratingly obtuse to accomplish that are relatively trivial in many other frameworks.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |