Log in Register FAQ Memberlist Search pcHDTV Forum Index
pcHDTV Forum

pcHDTV Forum Index -> General pcHDTV topics -> Question: Closed Caption? Goto page Previous  1, 2, 3, 4  Next
Post new topic  Reply to topic View previous topic :: View next topic 
PostPosted: Wed Sep 21, 2005 11:42 am Reply with quote
Scott Larson
 
Joined: 15 Oct 2003
Posts: 713
Location: Portland, OR




The XvMC problem turned out to be far more annoying than I had expected. The fundemental problem is that the XvMC code in mplayer wasn't really designed to display smooth video when a subpicture (either a menu or a caption) is being displayed over HD resolutions with lower-end MPEG accelerator chips. I'll have to tune mplayer's XvMC code to handle this better.

mplayer does a great job of not dropping frames by keeping rendered surface in a queue before they're displayed. The size of this queue is limited by the number of surfaces the hardware can hold (the FX5200 gives you eight). If your MPEG chip supports "backend rendering", you can use XvMCBlendSubpicture() to simply add the subpicture (caption or menu) to any surface already rendered. Unfortunately some MPEG hardware (i.e. my FX5200) only supports "frontend rendering" which means you have to find a another free surface to render the blended surface plus subpicture to. Since mplayer likes to keep the queue of rendered surfaces full, sometimes a free surface isn't available so the code has to wait for one. This is probably what's causing the video to be jumpy whenever there's a caption on the screen. The whole queuing system doesn't look like it expects to have to do this very often.

xine doesn't appear to use any XvMC subpicture support at all and simply uses the CPU to blend the subpicture directly into the rendered surface just like it does with xv.
View user's profile Send private message
PostPosted: Wed Sep 21, 2005 12:38 pm Reply with quote
Zoomy
Guest
 




Hey Scott..

you're doing a super job. Do you have a patch for us to try out yet? As I don't think it will make it into mplayer cvs for quite a while due to their big server-crash, their development has almost halted.

As for caption sync problem.. you probably are aware of this but there is already code in mplayer to display subtitles sooner/later so can't you rip parts of that code and use it with your captions? There is controls to change the delay on the fly with the keyboard (on default I think it's T or R -- something in that area)
PostPosted: Thu Sep 22, 2005 10:02 am Reply with quote
Scott Larson
 
Joined: 15 Oct 2003
Posts: 713
Location: Portland, OR




I think I can come up with a patch tomorrow night. I'll put everything in #ifdef EIA_708_CAPTIONS so you can def the code out if it causes problems. I haven't heard about their server crash. I pulled a version from cvs a couple of days ago and I still see patches coming in on their mailing list.

The code in spudec.c does solve exactly the same out-of-order problem my code solves but that code is for displaying DVD subtitles which are kooky bitmap graphics. EIA-608 is a totally different format, little more than ASCII and some screen control characters. DTVCC (the real HDTV captioning format) is also mostly ASCII with even more fancy control characters. There's no easy way to use the code in spudec.c for this.

The DVB-T standard supports the popular teletext format (much like EIA-608) but oddly they chose to add bitmap graphics like those on DVD's. It's not clear whether the standard requires both formats but most broadcasters probably will pass the teletext too since it's been popular for decades outside the U.S. It would be a shame to only have bitmap captions since text captions makes it easy for a tool to transcribe a show to a text file.

Last night I found a new problem: "Invasion" had both English and Spanish subtitles using both EIA-608 data channels. I've never seen anything use the other data channel before. The mplayer code doesn't know about Data Channel 2 control codes so it assumes the characters after them are just pop-on captions. I'll have to add some code to make it ignore them and maybe find a way to let the user select the two channels.
View user's profile Send private message
PostPosted: Fri Sep 23, 2005 10:37 pm Reply with quote
Scott Larson
 
Joined: 15 Oct 2003
Posts: 713
Location: Portland, OR




OK I think I have a patch here. It's against tonight's CVS.

To make it happen, apply the patch, put

#define EIA708_CAPTIONS

into your config.h and then make clean and recompile everything (never trust mplayer's Makefiles).

Then run mplayer on some transport stream file with the -subcc option and see what happens.

Remember that if you use XvMC, odds are it will retrace and stutter (it works fine with Xv), and a lot of stations just aren't very good with ATSC captions yet. Many captions could be three to five seconds behind the dialog. I know of no way to fix this since I can't display a caption that hasn't arrived yet.

I've found the best subtitles come with Fox's HD programming since they're inserted by the network, not the local station. It seems like someday all networks will do this.
View user's profile Send private message
PostPosted: Wed Sep 28, 2005 6:34 pm Reply with quote
dorphell
 
Joined: 01 Feb 2005
Posts: 71




Pretty neat.... that delay renders it useless but I know there's no easy fix for that. My stations are behind by at least 5 seconds.

I guess a quick hack would be to cache a certain amount but read subs on the fly. Have you thought about doing something like this?
View user's profile Send private message Visit poster's website
PostPosted: Thu Sep 29, 2005 3:14 pm Reply with quote
Scott Larson
 
Joined: 15 Oct 2003
Posts: 713
Location: Portland, OR




dorphell wrote:
Pretty neat.... that delay renders it useless but I know there's no easy fix for that. My stations are behind by at least 5 seconds.

You can contact the engineer of the station if their closed captioning isn't working well. FOX stations should have perfect captioning since it's inserted by the network. My CBS and NBC stations are very close too. My WB station is a mess. I've notified the C.E. of my UPN station that he's not sending any CCing during UPN programs. He said he wasn't aware of this and he's working on it right now.

Live events like football will always be behind since they're some poor person typing as fast as they can. Talk shows are also usually transcribed like live events even though they're taped -- it's cheaper.

Quote:
I guess a quick hack would be to cache a certain amount but read subs on the fly. Have you thought about doing something like this?

I have no idea what you mean.

There also seems to be a bug where a packet gets lost, dropping two characters: An example from Lost last night: "BUT BY THEN,E WERE ALREADY A THOUSAND MILES OFF COURSE."
View user's profile Send private message
PostPosted: Sat Oct 01, 2005 7:26 am Reply with quote
timh
 
Joined: 11 Feb 2005
Posts: 13
Location: Jacksonville, FL




People tend to forget that the original American Standard Code for Information Interchange (ASCII) is, in fact, only a 7-bit code.

If you want to handle an odd-parity 8-bit sequence rigorously, you should parity-check each character and insert a substitution character in place of characters that fail fail the parity check.

If the standard so specifies, use the designated subsitution character. If not, you're on your own and can select whatever you like.

Of course, what I'd expect is that the caption block is transmitted redundantly and with a CRC on the packet as a whole, making the whole idea of a parity bit pretty useless.
View user's profile Send private message
displaying subpictures
PostPosted: Sat Oct 01, 2005 7:44 pm Reply with quote
Guest
 




Scott Larson wrote:
The XvMC problem turned out to be far more annoying than I had expected. The fundemental problem is that the XvMC code in mplayer wasn't really designed to display smooth video when a subpicture (either a menu or a caption) is being displayed over HD resolutions with lower-end MPEG accelerator chips. I'll have to tune mplayer's XvMC code to handle this better.

xine doesn't appear to use any XvMC subpicture support at all and simply uses the CPU to blend the subpicture directly into the rendered surface just like it does with xv.




What about XDrawString to write the text directly to the display window?
PostPosted: Mon Oct 03, 2005 9:57 am Reply with quote
Scott Larson
 
Joined: 15 Oct 2003
Posts: 713
Location: Portland, OR




timh wrote:
If you want to handle an odd-parity 8-bit sequence rigorously, you should parity-check each character and insert a substitution character in place of characters that fail fail the parity check.

The mplayer code ignores parity at this point. In fact it ignores a whole heck of a lot of the EIA-608 standard. It didn't even support roll-up captions. The code is designed to make it difficult to support many EIA-608 control codes like italicized fonts since the code that displays the data expects a buffer with only ASCII characters.

Quote:
Of course, what I'd expect is that the caption block is transmitted redundantly and with a CRC on the packet as a whole, making the whole idea of a parity bit pretty useless.

In theory you shouldn't be getting byte parity errors in an MPEG frame's user data. The real problems start when you lose a frame or two and you lose characters. This may be what's happening. I can't think of any good way to indicate that data that was lost because actually most frames don't contain any caption data so it's possible no data was lost.
View user's profile Send private message
Re: displaying subpictures
PostPosted: Mon Oct 03, 2005 10:25 am Reply with quote
Scott Larson
 
Joined: 15 Oct 2003
Posts: 713
Location: Portland, OR




Anonymous wrote:
What about XDrawString to write the text directly to the display window?

XvMC surfaces are private data so you can't draw to them directly like a regular X window although it looks like you can draw to them without going through an XvMC routine like xine does. Both xine and mplayer have elaborate methods of adding text to the video depending on the method being used to display the video.
View user's profile Send private message
PostPosted: Mon Oct 03, 2005 1:15 pm Reply with quote
Guest
 




Scott Larson wrote:
timh wrote:
If you want to handle an odd-parity 8-bit sequence rigorously, you should parity-check each character and insert a substitution character in place of characters that fail fail the parity check.

The mplayer code ignores parity at this point. In fact it ignores a whole heck of a lot of the EIA-608 standard. It didn't even support roll-up captions. The code is designed to make it difficult to support many EIA-608 control codes like italicized fonts since the code that displays the data expects a buffer with only ASCII characters.

Not surprising. Back when ASCII was really ASCII, device-specific activities were typically indicated by use of the Shift-in/Shift-out ASCII control codes. Then escape sequences became popular, then whole new standards (generally 8-bit) took over. In some cases, instead of SI/SO, Device Control codes (DC1-DC4) were also used I believe. Unless those were for EBCDIC. It has been a while.

Quote:
Of course, what I'd expect is that the caption block is transmitted redundantly and with a CRC on the packet as a whole, making the whole idea of a parity bit pretty useless.

In theory you shouldn't be getting byte parity errors in an MPEG frame's user data. The real problems start when you lose a frame or two and you lose characters. This may be what's happening. I can't think of any good way to indicate that data that was lost because actually most frames don't contain any caption data so it's possible no data was lost.


Put another way, assuming CRCs on frames, the odds against a false positive CRC coming up are so high that there's little chance that enough data will be left intact to attempt to get by on parity-detection. So I agree.

And, since TS streams SHOULD be assuming noise and no error feedback, I /hope/ they're sending at least 2 of everything, but not having looked at any specs lately, I'm just stating my humble opinion.
PostPosted: Mon Oct 03, 2005 6:51 pm Reply with quote
Scott Larson
 
Joined: 15 Oct 2003
Posts: 713
Location: Portland, OR




Anonymous wrote:
And, since TS streams SHOULD be assuming noise and no error feedback, I /hope/ they're sending at least 2 of everything, but not having looked at any specs lately, I'm just stating my humble opinion.

No, the forward error correction happens at the hardware level (I think ATSC uses Reed-Solomon coding). MPEG-2 transport streams don't support error correction or duplicate packets and the bit rate wouldn't be fast enough for HD if stations had to send duplicate data.
View user's profile Send private message
Elaborate methods needed?
PostPosted: Wed Oct 05, 2005 12:06 am Reply with quote
Guest
 




Scott Larson wrote:
Anonymous wrote:
What about XDrawString to write the text directly to the display window?

XvMC surfaces are private data so you can't draw to them directly like a regular X window although it looks like you can draw to them without going through an XvMC routine like xine does. Both xine and mplayer have elaborate methods of adding text to the video depending on the method being used to display the video.


Elaborate methods sound like over-engineering, but is it needless over-engineering?

I guess you misunderstand where I'm going with XDrawString, or I misunderstand what type of data you have available for display.

If you have a text string, you can write the string directly to the display window itself, just like xawtv does it, and even use a custom font like xawtv uses, bypassing Xv or XvMC entirely.

As long as it's not the same color as the colorkey, the text should be visible, unless the image area is the same color as the text. Black background for the font would look blocky. It would look better if it were only a one pixel black border around each character outline, but hard to do? Outline fonts?

Another method you could use is the OSD function of the video card, but it's not as easy as XDrawString.

Am I not understanding the format of the text data you are trying to display: that it is not really text but something else that is only suitable for blending into the overlay video plane prior to display?

DTVCC looks like it uses YCrCb triplets. The data stream seems identical to DVDCC. Those can't be converted to text easily, so blending may be the only correct method.

Maybe there is no easy way out. You probably want to design a method that will work the same regardless of it being EIA-608 or EIA-708.

Thanks in advance for any clarification you can provide.
Re: Elaborate methods needed?
PostPosted: Wed Oct 05, 2005 12:55 pm Reply with quote
Scott Larson
 
Joined: 15 Oct 2003
Posts: 713
Location: Portland, OR




Anonymous wrote:
Am I not understanding the format of the text data you are trying to display: that it is not really text but something else that is only suitable for blending into the overlay video plane prior to display?

Well, if you can show where that XDrawString() call would go in mplayer or xine, I'd be interested. Both applications do everything with special OSD rendering functions and I guess it's for a reason. How do I keep a string on a display that's being rewritten thirty times a second?

Quote:
DTVCC looks like it uses YCrCb triplets. The data stream seems identical to DVDCC. Those can't be converted to text easily, so blending may be the only correct method.

No DTVCC is just text. It's mostly an enchancement of EIA-608 captioning and it's much simpler than DVD captioning.
View user's profile Send private message
Re: Elaborate methods needed?
PostPosted: Wed Oct 05, 2005 3:23 pm Reply with quote
Guest
 




Scott Larson wrote:
Anonymous wrote:
Am I not understanding the format of the text data you are trying to display: that it is not really text but something else that is only suitable for blending into the overlay video plane prior to display?

Well, if you can show where that XDrawString() call would go in mplayer or xine, I'd be interested. Both applications do everything with special OSD rendering functions and I guess it's for a reason. How do I keep a string on a display that's being rewritten thirty times a second?

Quote:
DTVCC looks like it uses YCrCb triplets. The data stream seems identical to DVDCC. Those can't be converted to text easily, so blending may be the only correct method.

No DTVCC is just text. It's mostly an enchancement of EIA-608 captioning and it's much simpler than DVD captioning.


DTVCC is just text with some formatting codes? Seems easily doable. It won't matter what's going on with the overlay side of video memory. Whatever you draw with XDrawString will stay there until you redraw it, much like how the OSD works, but with simpler X primitives for doing it. You could go nuts with the fonts, using any of the available X fonts. Italics, underlines, reverse video, colors all easily settable with X primitives and no OSD overhead.

You could even use a Tengwar font. That ought to irritate just about anyone but a Tolkien fan. Imagine watching Lord of the Rings on WB with Elvish Tengwar Closed Captioning. Would be awesome.

A font selection panel for the various DTVCC modes would eventually be needed for customization, but you could stick with a single font for each mode, or put the font names in the config file, until someone can get around to writing a font selection panel.

XDrawString only goes to the RGB colorspace, so it's completely isolated from the overlay. You could play Space Invaders on the RGB colorspace while DTV is displaying on the overlay colorspace, but you'd probably want to use colorkey value instead of black in that particular scenario, so you could see the DTV.

Where in mplayer or xine would it go? You have a better grasp of those programs than I do, from what I've been reading here on the forums. I don't have a clue. Maybe as OSD replacement functions. Call them RGB instead of OSD.

I may be mistaken, but I think OSD uses the YCrCb color values, so as to make it easier to store the DVD subpicture triplets directly to the OSD, as if it were specifically designed for DVD subpicture and not for CC.

You mentioned something about out-of-order and resorting to PTS. You might be able to depend on the MPEG temporal frame order to assemble the text, but you probably will have to use the PTS to know when to display it, or scroll it if necessary.

You can also use the temporal frame order for checking if any frames have dropped out. Not sure how you would handle that on display, maybe with spaces or some character to indicate something was missed.

The NTSC CC I've seen was like watching an old ASR33 teletype, because it was displaying the text so slowly and only in capitals. I haven't yet seen DTVCC so I don't know how fast the text updates.

I can understand the reasons for the elaborate methods for doing DVD subpicture blending, because of the nature of the DVD subpictures and the fact that some cards do not have OSD, but for CC text it seems like overkill, to me. I don't know if the EU has mandatory CC, but if they don't, it might explain why XDrawString never occured to our friends in Germany and Hungary and elsewhere in the world.

You might want to make sure the CC text doesn't overwrite the subpicture area, in case someone is playing a DVD that has both subtitles and CC, but as far as I can tell there are no subpictures for DTV. Not sure how you would go about reconciling that for DVD. Instruct the user to use one or the other? CC has descriptive text like [RAINDROPS FALLING] that are usually missing from DVD subpictures that have dialogue only.

It's pretty cool how the new video cards (GEF2+) have 3 different colorspaces to write to now, RGB, OSD and YCrCb.

By the way, thanks for working on this. My hearing is going and I'll be needing CC sooner than I hoped.
Question: Closed Caption?
  pcHDTV Forum Index -> General pcHDTV topics
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
All times are GMT - 7 Hours  
Page 2 of 4  
Goto page Previous  1, 2, 3, 4  Next
  
  
 Post new topic  Reply to topic  


Powered by phpBB © 2001-2003 phpBB Group
Theme created by Vjacheslav Trushkin