Recent changes to bugs

Access to C4 Erase Page flag

James D'Arcy — Wed, 30 Oct 2019 11:21:45 -0000

Hello VBI Team.

I hope this library is still somewhat active and that someone is out there.

I am trying to determine whenever a CAPTION CLEAR COMMAND event occurs in Australain broadcast streams and am wondering if someone can assist.

I am referring to the Australian Broadcast standards found at "https://www.freetv.com.au/wp-content/uploads/2019/08/OP-42-Closed-Captioning-Issue-5-May-2018.pdf"

Note the relevant section on the first page:

CAPTION CLEAR COMMAND
A Command to clear any existing caption shall be included in the first two (2) seconds of
a program. A Command to clear shall also be included two (2) seconds before the end
of a program.
A command to clear any existing caption shall be included in the first half (0.5) second of
a commercial. Where closed caption data is included, the caption information should
commence no earlier than a half (0.5) second after the start of active video and a caption
erase signal is required (last caption time code out) not later than a half (0.5) second
before the end of active video. First caption time code in will be at an arbitrary point
determined by content.

First of all, I believe I am correct in believing that the "Caption Clear Command" referred to in the above documents is the "C4 Erase Page" flag (Byte 9, bit 8). Is that correct? If not, then I need to find another path.

So, given that that is correct, my goal was to find when this flag was being set. There appeared to be no direct way served up via the zvbi API as it looked at first that VBI header details are not directly available from within the library.

But then I noticed some comments in the documentation for the 'VBI_EVENT_TTX_PAGE' event:

If any of the roll_header, header_update or clock_update flags
are set ev.ttx_page.raw_header is a pointer to the raw header data
(40 bytes), which remains valid until the event handler returns.

so I thought all I would need to do is wait until the 'header_update' flag is set and then look at ev.ttx_page.raw_header[8] and see if the C4_ERASE_PAGE bit is set (the eighth bit I think)

However, the header_update flag is never set.

I have also tried working with the vbi_pfc_demux_new() and vbi_pfc_demux_feed_frame() calls to see if I can get a VbiPageFunctionClearCallback method but the callback is never called.

Further, I only ever get VBI_EVENT_TTX_PAGE events, no matter what I set the vbi_event_handler_register flags to.

I have had a detailed look at the ffmpeg, vlc and gstreamer code but none of them are going to the level of detail I am after.

Can someone point me in the correct direction, perhaps with some sample code please?

Kind regards

James

#203 Handle page termination with same page number

Stefan Pöschel — Mon, 23 Oct 2017 09:49:56 -0000

Ah, interesting....I originally thought that there was just a termination with a page header of the same page and that this case was merely not considered by the spec (but it is indeed, as you said). So that this is a corner case where it would make sense to add a workaround to libzvbi.

But there is no immediate termination; there is only a page header some seconds later with Erase Page set that removes the displayed subtitles. So my attached patch doesn't make sense here. And the transmission clearly doesn't comply to the Teletext spec.

For me this is something that definitely has to be fixed on broadcaster side. Hence I withdraw my request to fix this issue in libzvbi (as I'm a strong fan of fixing the source of an issue, if possible).

But I don't know the overall intention of libzvbi: Whether to only accept conform transmissions or to also cope with buggy encoders. So if you want to, you can fix this of course ;-)

#203 Handle page termination with same page number

Michael H. Schimek — Mon, 23 Oct 2017 07:00:39 -0000

Hi Stefan,

EN 300 706 Annex A.1 recommends to insert a filler/page termination header with page number 0xFF if only a single page is transmitted, to commit the page to memory as well as display subtitles immediately rather than at the reception of the next page header, which may not occur until the subtitles are changing again. The libzbvi decoder recognizes 0xFF headers, but as you realized, not their omission. The check was added as a work-around for some stations repeating headers as filler, and causing a number of undesirable effects such as unnecessary page update events and redrawing, visible blanking if a header with an erase flag repeats ahead of the page body, or duplicate lines in a subtitle transcript.

I would prefer to keep the check but accept repeated headers as terminators under certain conditions, such as a different subpage number, the presence of a page body, or a page erase and subtitle flag in the suspect header. Preliminary tests with my Teletext samples were promising. Further I propose a new auto-termination feature where the decoder delivers subtitle events on its own if no terminator is received within a reasonable time. I already wrote a regression test failing on missing and delayed subtitles and will try auto-termination shortly.

#203 Handle page termination with same page number

Stefan Pöschel — Fri, 20 Oct 2017 14:48:34 -0000

I just wanted to ask if there is anything new regarding this or possibly further information missing.

Handle page termination with same page number

Stefan Pöschel — Wed, 24 May 2017 19:59:22 -0000

I recently had a look at two VLC issues that affect Teletext subtitles and libzvbi:
https://trac.videolan.org/vlc/ticket/13498
https://trac.videolan.org/vlc/ticket/18113

In both cases, the subtitle lines remain displayed until being replaced by a new version of the specific line. So the subtitles never disappear and often an old line is still displayed, after another, more recent line is displayed/updated.

At first I suspected that the Erase Page Flag (C4) in the Page Header was not set, but the flag was indeed set (at least once between two LOPs with subtitles). After further debugging it turned out that the Teletext service in the two issues consists of just a single Teletext page - which is the reason here:

The Teletext spec defines the Magazine Serial Flag (C11) in Table 2 (ETSI EN 300 706 v1.2.1) as (bold font by me):

When set to '1' the service is designated to be in Serial mode and the transmission of a page is terminated by the next page header with a different page number.
When set to '0' the service is designated to be in Parallel mode and the transmission of a page is terminated by the next page header with a different page number but the same magazine number.
The same setting shall be used for all page headers in the service.
(BTW it doesn't really matter, but in these two cases this flag is always 0)

In vbi_decode_teletext in /src/packet.c, storing the page is aborted, if the condition "page terminated" in the C11 definition is not met. But as the service here consists of only one page, this condition is never met!

On the one hand, I am a great fan of implementing strictly according to the specs. On the other hand I think that the case of broadcasting just one Teletext page have by mistake not been considered when creating the spec. Furthermore I could not think of a case where removing the check for this condition could harm the decoding (I'm not sure, but I don't know why encoders should do that....which is the reason why I post this into Issues and not into Patches). Hence I removed the check for page termination (for both serial/parallel case) that aborts storing the page. This solves the problem for me for the two files.

Please find the patch attached. I did not check if further modifications are necessary or if the change has other implications. I also did not add a comment to the code (however it could make sense) as I'm not sure how you like to word such special case explanations.

#202 test-pdc.h:92: bad sizeof ?

Michael H. Schimek — Mon, 17 Oct 2016 20:53:01 -0000

status: open --> closed-fixed
assigned_to: Michael H. Schimek
Group: -->

test-pdc.h:92: bad sizeof ?

dcb — Thu, 13 Oct 2016 07:29:35 -0000

test-pdc.h:92:45: warning: argument to 'sizeof' in 'int memcmp(const void, const void, size_t)' call is the same expression as the first source; did you mean to dereference it? [-Wsizeof-pointer-memaccess]

Source code is

    return (0 == memcmp (this, &other, sizeof (this)));

Maybe better code

    return (0 == memcmp (this, &other, sizeof (*this)));

test-pdc.h:92: bad sizeof ?

dcb — Thu, 13 Oct 2016 07:29:35 -0000

Ticket 202 has been modified: test-pdc.h:92: bad sizeof ?
Edited By: Michael H. Schimek (mschimek)
Status updated: u'open' => u'closed-fixed'
Owner updated: None => u'mschimek'

zvbi compilation warnings

Ilya Gordeev — Sun, 20 Apr 2014 18:36:39 -0000

Building zvbi-0.2.35 by gcc-4.8.2 I'v got this compilation warnings:
src/xds_demux.c:829:29: warning: iteration 24u invokes undefined behavior [-Waggressive-loop-optimizations]
contrib/ntsc-cc.c:1164:24: warning: iteration 25u invokes undefined behavior [-Waggressive-loop-optimizations]
contrib/atsc-cc.c:4076:28: warning: iteration 25u invokes undefined behavior [-Waggressive-loop-optimizations]

Propose patch (in attachment), if I understand that code correctly.

#199 [packet.c:1943]: (warning) Logical disjunction always evaluates to true: function != 4 || function != 5.

Michael H. Schimek — Tue, 18 Feb 2014 17:06:51 -0000

status: open --> closed-fixed
assigned_to: Michael H. Schimek
Group: -->