OutOfMemory error when parsing

Martin Jericho — Sun, 29 Sep 2024 04:47:33 -0000

Hi Andy. Thanks for reporting the issue.

I see what you mean about sourceforge. I just noticed they removed all of the documentation from my project's website a couple of months ago without notification. I just fixed that. But no I don't have any intention of moving the project to github at this point in time.

Firstly, you might like to try using the latest DEV version 3.5. There have been a few improvements and bug fixes to the Renderer class. You can download it here:
http://jericho.htmlparser.net/temp/jericho-html-3.5-dev.zip
The 3.5-dev version is always a release candidate and can be used as a reliable substitute for the last official 3.4 release.

According to the release notes there are no bug fixes that look related to the issue you're experiencing, but it's worth a try.

Another thing, you don't need the following line, as Source is already a subclass of Segment. Just use new Renderer(htmlSource) instead.
Segment htmlSeg = new Segment(htmlSource, 0, htmlSource.length());

If using 3.5-dev doesn't solve your problem, send me a link to the problematic source document and I'll take a look.

Let me know how it goes.

Module System support

Martin Jericho — Mon, 21 Aug 2023 08:38:51 -0000

Hi Ethan,

Thank you for the suggestion. Yes I got a request for this already last year:
https://sourceforge.net/p/jerichohtml/bugs/93/

The biggest barrier at the moment is the fact that I implemented a new major feature a few years ago (a web crawler API) but it remains poorly documented, and could probably use a couple of minor enhancements before it is officially released. That means all bug fixes since then have just gone into the DEV release:
http://jericho.htmlparser.net/temp/jericho-html-3.5-dev.zip

The DEV release is very stable, with no known bugs, and recommended for production use. I just haven't released it officially yet because of the missing documentation.

That DEV download probably includes updates that I never committed to the Bazaar repository, so you should consider that one the latest version. In fact I don't even remember why I bothered with a source code repository in the first place, as external contributions have been negligible.

Unfortunately due to other priorities I simply don't have time to look at doing a new official release. It might be time to hand the whole thing over to someone else, but not sure if anyone would be interested. And doing a handover would no doubt involve more work than an official release. So the project is sort of in limbo.

This library doesn't actually have any external dependencies, so one work-around would be to just add the source code to your own project. If you notice a bug you can always just file sync your copy with the one in the DEV zip file. Bug fixes these days are extremely rare and minor. Would that approach work for you?

Cheers
Martin

Jericho 3.4/3.5-dev parsing bug(?)

Martin Jericho — Wed, 07 Sep 2022 01:41:31 -0000

P.S. When you want to include HTML in your post, you need to enclose it in a code block, otherwise the HTML is parsed and doesn't show properly.

For example, your sample document should look like this:

<html>
  <head>
    <meta http-equiv=&quot;Content-Type&quot; content=&quot;html; charset=UTF-8&quot;>
  </head>
</html>

Jericho 3.4/3.5-dev parsing bug(?)

Martin Jericho — Wed, 07 Sep 2022 01:38:16 -0000

Hi Davy,

The sample HTML you are feeding it doesn't specify a valid encoding and is therefore parsed correctly. Because the quotes are encoded, they are included in the value of the content attribute, which is why the end quote is interpreted as part of the encoding name.

You say that the sample content occurs when it is "inserted into an iframe". I assume you mean it appears as the value of the iframe srcdoc attribute.

In that case, your sample document should be the HTML containing the iframe, not the value of the srcdoc attribute in isolation.

If you run the parent HTML document through the parser, it should correctly pick up the encoding specified in the parent HTML document, and not try to interpret the encoding in the srcdoc attribute.

If you want to parse the child document HTML from the iframe in isolation, you need to decode the value of the srcdoc attribute first. Then the parser will correctly detect the encoding of the child document.

I hope that makes sense!

Cheers
Martin

Jericho removing Button elements

Remi Rosenthal — Wed, 29 Jun 2022 10:35:09 -0000

Hi Martin,

Thanks a lot for patching this. I now get expected behaviour!

Kind regards,
Remi

Jericho removing Button elements

Martin Jericho — Wed, 29 Jun 2022 10:17:02 -0000

Hi Remi,

I didn't document anywhere why I made the decision to remove the content of button elements. In general I was copying the behaviour of how some email clients create pure text versions of HTML emails. Maybe I just thought they should be removed because all other form elements (INPUT, TEXTAREA etc) are removed. Or maybe I just didn't think much about it!

I've modified the Render class in version 3.5 to include the content of BUTTON elements.

Until version 3.5 is officially released, the development version is available here:
http://jericho.htmlparser.net/temp/jericho-html-3.5-dev.zip

The development version is always pretty much as stable as an official release. It has been a long time since an official release because the new WebBot functionality hasn't been documented yet, and I simply don't have time to work on it.

Cheers
Martin

Jericho removing Button elements

Remi Rosenthal — Wed, 29 Jun 2022 09:25:47 -0000

Hi,

I've noticed that the Jericho Renderer doesn't include Button elements in its toString(). This is presumably because button is mapped to a RemoveElementHandler in Renderer.
I would be interested to hear the rationale behind this, but more importantly, is there a way to override this behaviour on my end?

You can reproduce with something as simple as:
<button>My Button</button>
Which will result in an empty string.

Many thanks

3.5

Martin Jericho — Sun, 20 Feb 2022 16:44:58 -0000

Hi Andrew,

The release.txt file does mention "minor changes to Renderer behaviour" for version 3.5. The new behaviour is more consistent with browser behaviour so it is most likely an intended change.

Cheers
Martin

3.5

Andrew Smith — Fri, 04 Feb 2022 15:09:36 -0000

Hi Martin,

Thanks for responding so quickly. Since my last message, I've been trying out 3.5-dev as I was hoping to take advantage of the memory consumption improvements, but have come across a behaviour difference for the Renderer between 3.4 and 3.5.

For example, <p>Hello</p><p><br></p><p>There</p> used to output Hello\r\n\r\nThere
But now in 3.5-dev it outputs Hello\r\n\r\n\r\nThere

Is this an expected behaviour change? I have attached a screenshot of the Renderer configured

3.5

Martin Jericho — Wed, 02 Feb 2022 15:48:13 -0000

Hi Andrew. Version 3.5 hasn't been officially released yet because the newest feature, a web crawler API, has not been fully documented yet.
The project is not dead, and minor improvements continue to make their way into the DEV version, but other time commitments have prevented the completion of the documentation and an official release for years.
The 3.5-dev version (http://jericho.htmlparser.net/temp/jericho-html-3.5-dev.zip) is always a release candidate and can be used as a reliable substitute for the last official 3.4 release.

Recent posts to Discussion

OutOfMemory error when parsing

Module System support

Jericho 3.4/3.5-dev parsing bug(?)

Jericho 3.4/3.5-dev parsing bug(?)

Jericho removing Button elements

Jericho removing Button elements

Jericho removing Button elements

3.5

3.5

3.5