Your Benchmark is not meaningful.
I wrote a microbenchmark for this three libraries and got as result, that there is no significant difference.
Benchmark Mode Cnt Score Error Units
HttpBenchmark.httpClientGoogle avgt 2 151.162 ms/op
HttpBenchmark.httpClientStackoverflow avgt 2 151.086 ms/op
HttpBenchmark.httpUrlConnectionGoogle avgt 2 235.869 ms/op
HttpBenchmark.httpUrlConnectionStackoverflow avgt 2 145.162 ms/op
HttpBenchmark.jsoupGoogle avgt 2 391.162 ms/op
HttpBenchmark.jsoupStackoverflow avgt 2 188.059 ms/op
There are only one small difference between your tests and mine:
- JSoup set header "Accept-Encoding", "gzip" this will reduce bandwidth
- JSoup uses an bigger buffer (32kb)
- Reuse HttpClient is needed
In my tests JSoup is the slowest. Of course only JSoup parses the response.
My Benchmark:
@Warmup(iterations = 1, time = 3, timeUnit = TimeUnit.SECONDS)
@Measurement(iterations = 2, time = 5, timeUnit = TimeUnit.SECONDS)
@Fork(1)
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MILLISECONDS)
@State(Scope.Benchmark)
@Threads(1)
public class HttpBenchmark {
private static final String GOOGLE = "https://google.com/";
private static final String STACKOVERFLOW = "https://stackoverflow.com";
private final CloseableHttpClient httpClient = HttpClientBuilder.create().build();
@Benchmark
public void httpClientGoogle() throws Exception {
httpClient(GOOGLE);
}
@Benchmark
public void httpClientStackoverflow() throws Exception {
httpClient(STACKOVERFLOW);
}
@Benchmark
public void httpUrlConnectionGoogle() throws Exception {
httpUrlConnection(GOOGLE);
}
@Benchmark
public void httpUrlConnectionStackoverflow() throws Exception {
httpUrlConnection(STACKOVERFLOW);
}
@Benchmark
public void jsoupGoogle() throws Exception {
jsoup(GOOGLE);
}
@Benchmark
public void jsoupStackoverflow() throws Exception {
jsoup(STACKOVERFLOW);
}
private void httpClient(final String url) throws Exception {
final CloseableHttpResponse response = httpClient.execute(new HttpGet(url));
final BasicResponseHandler basicResponseHandler = new BasicResponseHandler();
basicResponseHandler.handleResponse(response);
response.close();
}
private void httpUrlConnection(final String url) throws Exception {
final HttpURLConnection httpURLConnection = (HttpURLConnection) new URL(url).openConnection();
httpURLConnection.addRequestProperty("Accept-Encoding", "gzip");
try (final BufferedInputStream r = new BufferedInputStream(httpURLConnection.getInputStream())) {
final byte[] tmp = new byte[1024 * 32];
int read;
while (true) {
read = r.read(tmp);
if (read == -1) {
break;
}
}
}
}
private void jsoup(final String url) throws Exception {
Jsoup.connect(url).execute().parse();
}
}
Could you please tell me what you used to perform these tests ? I want to do more tests with it
I found out it is JMH, I'll do more tests, thanks for the idea of benchmarking
Great, it would be awesome if you could accept my answer
Even by testing more cases I found out same results, Jsoup is faster than HUC for most websites I tested, I think this is duo to some optimization done by Jsoup for HUC, for example connection persisting (cause while testing, many requests are sent to a same website) which I was not using in my HUC test, but not sure JSoup is using it