[vdr] French DVB-T Channel IDs vs. EIT Channel IDs

[Patrick Boettcher]

Hi list,

since some time (1-2 years) I noticed that the DVB-T channels do not
show EPG anymore. I'm not sure what is the root-cause: either it
was the migration of French DVB-T to HD (in 2016) or an update of VDR.

Anyway, this morning I took the time to figure out what is wrong and
here is what I found.

In short, the channel-ID generated from the one received in the EIT
does not correspond to a known channel-ID from the channel list, and
VDR drops the whole section:

In cEIT::cEIT (from eit.c) we see

tChannelID channelID(Source, getOriginalNetworkId(), getTransportStreamId(), getServiceId());
cChannel *Channel = Channels->GetByChannelID(channelID, true);

and channel is NULL in my case.

I added some debug prints to GetByChannelID() in the search loop:

printf("sid: %d == %d, %s, %s %d\n",
Channel->Sid(),
sid,
(const char *)ChannelID.ToString(),
(const char *)Channel->GetChannelID().ToString(),
Channel->GetChannelID() == ChannelID);

And here is an example of what I get:

sid: 517 == 517, T-8442-2-517, T-0-506-517 0

The internal channel ID of vdr reads T-0-506-517. 517 is the SID, 506
is the radio-channel frequency in MHz. This is the right internal
channel for this EIT-section.

The EIT-channel-ID is telling me that 8442 is the original network ID
and 2 might be the radio-channel ID.

I tweaked the operator==() of ChannelID to make it work, but not in a
contributable manner.

How should this be fixed correctly?

Is this a regression introduced somewhere?

Can it be solved by configuration?

Thanks for any help.

best regards,
--
Patrick.

[Patrick Boettcher]

Hi Klaus,

thank you for your reply.

On Mon, 26 Feb 2018 16:58:20 +0100

Post by Patrick Boettcher
How should this be fixed correctly?
Is this a regression introduced somewhere?
Can it be solved by configuration?

where did the channel with ID T-0-506-517 come from in the first
place?

I ran dvbv5-scan with -O vdr and it gave me the line. I manually
copied it to channels.conf. Upon vdr start I saw that VDR was updating
the PIDs.

Here's what came out of it:

France 2:586000:B8C23G32M64S0T8Y0:T:0:120=27:0; \
130=***@122,131=***@122,132=***@122:0;140=fra,142=fra :0:257:0:0:0

(\ and line break added by me)

What happens if you delete that channel from VDR's channel list and
tune to the transponder? Does it create the channel with the correct
ID?

Good idea.

I deleted the channel and switched to another channel on the same
transponder.

Nothing happens. VDR does not re-"find" the channel. VDR is set up to
"Add New Transponders".

Running scan -c (current transponder) gives:

0x0000 0x0101: pmt_pid 0x006e GR1 B -- France 2 (running)
0x0000 0x0104: pmt_pid 0x0136 GR1 B -- France 4 (running)
0x0000 0x0105: pmt_pid 0x01fe GR1 B -- France Ô (running)
0x0000 0x0106: pmt_pid 0x0262 GR1 B -- franceinfo: (running)
0x0000 0x0111: pmt_pid 0x00d2 GR1 -- F3 Paris Ile-de-France (running)
0x0000 0x0170: pmt_pid 0x02c6 R1 TFL -- BFM Paris (running)
dumping lists (6 services)
France 2 (0x0101) 01: PCR == V V 0x0078 SUB 0x008e
France 4 (0x0104) 01: PCR == V V 0x0140 SUB 0x0156
France Ô (0x0105) 01: PCR == V V 0x0208 SUB 0x021c
franceinfo (0x0106) 01: PCR == V V 0x026c SUB 0x0280
F3 Paris Ile-de-France (0x0111) 01: PCR == V V 0x00dc A 0x00e6 (fre) 0x00e7 (qad) 0x00e8 (qaa) SUB0x00f2
BFM Paris (0x0170) 01: PCR == V V 0x02d0

(attached is the complete output with -vvv.)

Any idea?

--
Patrick.

[Patrick Boettcher]

On Tue, 27 Feb 2018 16:54:35 +0100

Hello Patrick,
your scan doesn't contain the NIT, which is where the transponder
innformation is broadcast. Can you scan that, too, please?

Hi Klaus,

NIT attached (from another multiplex with the same symptoms).

best regards,
--
Patrick.

[Patrick Boettcher]

On Wed, 28 Feb 2018 11:01:23 +0100

Post by Patrick Boettcher
On Tue, 27 Feb 2018 16:54:35 +0100

Hello Patrick,
your scan doesn't contain the NIT, which is where the transponder
innformation is broadcast. Can you scan that, too, please?

Hi Klaus,
NIT attached (from another multiplex with the same symptoms).

DVB-DescriptorTag: 90 (0x5a) [=
terrestrial_delivery_system_descriptor] descriptor_length: 11 (0x0b)
Center frequency: 0xffffffff (= 42949672.095 kHz)
@@ -219,6 +219,8 @@
cDvbTransponderParameters dtp;
int Source = cSource::FromData(cSource::stTerr);
int Frequency = Frequencies[0] =
sd->getFrequency() * 10;
+ Frequency = Transponder() * 1000000;//XXX
+ dsyslog("Frequency = %08X, Transponder = %d",
sd->getFrequency(), Transponder());//XXX static int Bandwidths[] =
{ 8000000, 7000000, 6000000, 5000000, 0, 0, 0, 0 };
dtp.SetBandwidth(Bandwidths[sd->getBandwidth()]); static int
Constellations[] = { QPSK, QAM_16, QAM_64, QAM_AUTO };

It works. I had to have VDR recreate the channels because it was mixing
things up with the already existing ones.

As to why the frequency is set to "-10" I don't know (yet), but I
remember it has always been like this since 2005.

Initial tuning files have always contained all active frequencies for
their region, because the NIT did not set the center frequencies
correctly.

It might be because the multiplexed transport streams are generated
centrally and then distributed to all the base-stations where they are
emitted as-is but on different frequencies depending on the region.

What can we do to get this upstream?

Thanks for your help,
--
Patrick.

[Patrick Boettcher]

On Thu, 1 Mar 2018 10:22:10 +0100

Post by Patrick Boettcher
On Wed, 28 Feb 2018 11:01:23 +0100

Post by Patrick Boettcher
On Tue, 27 Feb 2018 16:54:35 +0100

Hello Patrick,
your scan doesn't contain the NIT, which is where the transponder
innformation is broadcast. Can you scan that, too, please?

Hi Klaus,
NIT attached (from another multiplex with the same symptoms).

DVB-DescriptorTag: 90 (0x5a) [=
terrestrial_delivery_system_descriptor] descriptor_length: 11 (0x0b)
Center frequency: 0xffffffff (= 42949672.095 kHz)
@@ -219,6 +219,8 @@
cDvbTransponderParameters dtp;
int Source = cSource::FromData(cSource::stTerr);
int Frequency = Frequencies[0] =
sd->getFrequency() * 10;
+ Frequency = Transponder() * 1000000;//XXX
+ dsyslog("Frequency = %08X, Transponder = %d",
sd->getFrequency(), Transponder());//XXX static int Bandwidths[] =
{ 8000000, 7000000, 6000000, 5000000, 0, 0, 0, 0 };
dtp.SetBandwidth(Bandwidths[sd->getBandwidth()]); static int
Constellations[] = { QPSK, QAM_16, QAM_64, QAM_AUTO };

It works. I had to have VDR recreate the channels because it was
mixing things up with the already existing ones.

I called victory too early: something (channel-updates?) is overwriting
the frequencies and modulation-data for all DVB-T-channels.

Please see the attached log-file.

--
Patrick.

[Patrick Boettcher]

On Thu, 1 Mar 2018 10:22:10 +0100

Post by Patrick Boettcher
On Wed, 28 Feb 2018 11:01:23 +0100

Post by Patrick Boettcher
On Tue, 27 Feb 2018 16:54:35 +0100

Hello Patrick,
your scan doesn't contain the NIT, which is where the transponder
innformation is broadcast. Can you scan that, too, please?

Hi Klaus,
NIT attached (from another multiplex with the same symptoms).

DVB-DescriptorTag: 90 (0x5a) [=
terrestrial_delivery_system_descriptor] descriptor_length: 11 (0x0b)
Center frequency: 0xffffffff (= 42949672.095 kHz)
@@ -219,6 +219,8 @@
cDvbTransponderParameters dtp;
int Source = cSource::FromData(cSource::stTerr);
int Frequency = Frequencies[0] =
sd->getFrequency() * 10;
+ Frequency = Transponder() * 1000000;//XXX
+ dsyslog("Frequency = %08X, Transponder = %d",
sd->getFrequency(), Transponder());//XXX static int Bandwidths[] =
{ 8000000, 7000000, 6000000, 5000000, 0, 0, 0, 0 };
dtp.SetBandwidth(Bandwidths[sd->getBandwidth()]); static int
Constellations[] = { QPSK, QAM_16, QAM_64, QAM_AUTO };

It works. I had to have VDR recreate the channels because it was
mixing things up with the already existing ones.
As to why the frequency is set to "-10" I don't know (yet), but I
remember it has always been like this since 2005.

Here is the explanation why the frequency field in French DVB-T shall
not be used, (cf. CSA-Profil-de-signalisationv3.4.pdf, page 30,
terrestrial_delivery_system_descriptor):

le terrestrial delivery_system_descriptor diffusés dans la NIT
correspond au cas général. En effet, la NIT ne décrit pas de façon
exhaustive le réseau actuel d’émetteurs, mais l’organisation des
services des multiplex diffusés : ainsi les fréquences des multiplex
renseignées par le paramètre centre_frequency ont une valeur fixée à
0xFFFFFFFF. Ces fréquences ne sont pas à prendre en compte

It says, grosso modo, that the NIT does not describe the network in an
exhaustive manner, but it does for services. Le field centre_frequency
contains the value 0xFFFFFFFF and is not to be taken into consideration.

--
Patrick.

[Patrick Boettcher]

On Thu, 1 Mar 2018 11:38:24 +0100

Post by Patrick Boettcher

DVB-DescriptorTag: 90 (0x5a) [=
terrestrial_delivery_system_descriptor] descriptor_length: 11
(0x0b) Center frequency: 0xffffffff (= 42949672.095 kHz)
@@ -219,6 +219,8 @@
cDvbTransponderParameters dtp;
int Source = cSource::FromData(cSource::stTerr);
int Frequency = Frequencies[0] =
sd->getFrequency() * 10;
+ Frequency = Transponder() * 1000000;//XXX
+ dsyslog("Frequency = %08X, Transponder = %d",
sd->getFrequency(), Transponder());//XXX static int Bandwidths[] =
{ 8000000, 7000000, 6000000, 5000000, 0, 0, 0, 0 };
dtp.SetBandwidth(Bandwidths[sd->getBandwidth()]); static int
Constellations[] = { QPSK, QAM_16, QAM_64, QAM_AUTO };

It works. I had to have VDR recreate the channels because it was
mixing things up with the already existing ones.
As to why the frequency is set to "-10" I don't know (yet), but I
remember it has always been like this since 2005.

Can you please send me the log line that was generated by the added
dsyslog() statement?

It does:

Mar 1 11:28:43 vdr vdr: [4096] Frequency = FFFFFFFF, Transponder = 586

--
Patrick.

[Patrick Boettcher]

On Fri, 9 Mar 2018 11:30:36 +0100

Post by Patrick Boettcher
On Wed, 28 Feb 2018 11:01:23 +0100

Post by Patrick Boettcher
On Tue, 27 Feb 2018 16:54:35 +0100

Hello Patrick,
your scan doesn't contain the NIT, which is where the
transponder innformation is broadcast. Can you scan that, too,
please?

Hi Klaus,
NIT attached (from another multiplex with the same symptoms).

DVB-DescriptorTag: 90 (0x5a) [=
terrestrial_delivery_system_descriptor] descriptor_length: 11
(0x0b) Center frequency: 0xffffffff (= 42949672.095 kHz)
@@ -219,6 +219,8 @@
cDvbTransponderParameters dtp;
int Source = cSource::FromData(cSource::stTerr);
int Frequency = Frequencies[0] =
sd->getFrequency() * 10;
+ Frequency = Transponder() * 1000000;//XXX
+ dsyslog("Frequency = %08X, Transponder = %d",
sd->getFrequency(), Transponder());//XXX static int Bandwidths[] =
{ 8000000, 7000000, 6000000, 5000000, 0, 0, 0, 0 };
dtp.SetBandwidth(Bandwidths[sd->getBandwidth()]); static int
Constellations[] = { QPSK, QAM_16, QAM_64, QAM_AUTO };

It works. I had to have VDR recreate the channels because it was
mixing things up with the already existing ones.
As to why the frequency is set to "-10" I don't know (yet), but I
remember it has always been like this since 2005.
Initial tuning files have always contained all active frequencies
for their region, because the NIT did not set the center frequencies
correctly.
It might be because the multiplexed transport streams are generated
centrally and then distributed to all the base-stations where they
are emitted as-is but on different frequencies depending on the
region.
What can we do to get this upstream?

I'm afraid I don't see how this case can be handled correctly.
Maybe somebody else has an idea?

One approach might be to ignore frequencies for DVB-T channels update
during NIT parsing (in France only?).

From my experiences with DVB-T SetTopBoxes using the NIT for
new channel-discovery is rarely done. No customer of my ex-employer was
using it - scanning was entirely based on continuous frequency-band
scanning with a spare demod.

That would be my idea. But I don't know whether this can be easily
applied to VDR.

--
Patrick.

[Patrick Boettcher]

On Fri, 9 Mar 2018 11:55:02 +0100

Post by Patrick Boettcher
On Fri, 9 Mar 2018 11:30:36 +0100

Post by Patrick Boettcher
On Wed, 28 Feb 2018 11:01:23 +0100

Post by Patrick Boettcher
On Tue, 27 Feb 2018 16:54:35 +0100

Hello Patrick,
your scan doesn't contain the NIT, which is where the
transponder innformation is broadcast. Can you scan that,
too, please?

Hi Klaus,
NIT attached (from another multiplex with the same
symptoms).

DVB-DescriptorTag: 90 (0x5a) [=
terrestrial_delivery_system_descriptor] descriptor_length: 11
(0x0b) Center frequency: 0xffffffff (= 42949672.095 kHz)
@@ -219,6 +219,8 @@
cDvbTransponderParameters dtp;
int Source =
cSource::FromData(cSource::stTerr); int Frequency =
Frequencies[0] = sd->getFrequency() * 10;
+ Frequency = Transponder() * 1000000;//XXX
+ dsyslog("Frequency = %08X, Transponder = %d",
sd->getFrequency(), Transponder());//XXX static int
Bandwidths[] = { 8000000, 7000000, 6000000, 5000000, 0, 0, 0,
0 }; dtp.SetBandwidth(Bandwidths[sd->getBandwidth()]); static
int Constellations[] = { QPSK, QAM_16, QAM_64, QAM_AUTO };

It works. I had to have VDR recreate the channels because it was
mixing things up with the already existing ones.
As to why the frequency is set to "-10" I don't know (yet), but I
remember it has always been like this since 2005.
Initial tuning files have always contained all active frequencies
for their region, because the NIT did not set the center
frequencies correctly.
It might be because the multiplexed transport streams are
generated centrally and then distributed to all the
base-stations where they are emitted as-is but on different
frequencies depending on the region.
What can we do to get this upstream?

I'm afraid I don't see how this case can be handled correctly.
Maybe somebody else has an idea?

One approach might be to ignore frequencies for DVB-T channels
update during NIT parsing (in France only?).

From my experiences with DVB-T SetTopBoxes using the NIT for

new channel-discovery is rarely done. No customer of my ex-employer
was using it - scanning was entirely based on continuous
frequency-band scanning with a spare demod.
That would be my idea. But I don't know whether this can be easily
applied to VDR.

Well, maybe this works for you?
--- nit.c 2016/12/23 14:16:59 4.4
+++ nit.c 2018/03/09 10:53:38
@@ -218,6 +218,8 @@
SI::TerrestrialDeliverySystemDescriptor *sd =
(SI::TerrestrialDeliverySystemDescriptor *)d;
cDvbTransponderParameters dtp; int Source =
cSource::FromData(cSource::stTerr);
+ if (sd->getFrequency() == 0xFFFFFFFF)
+ continue;
int Frequency = Frequencies[0] =
sd->getFrequency() * 10; static int Bandwidths[] = { 8000000,
7000000, 6000000, 5000000, 0, 0, 0, 0 };
dtp.SetBandwidth(Bandwidths[sd->getBandwidth()]);

This is what I'm doing right now (though I'm using break instead of
continue).

But won't his inhibit the update of modulation parameters of the
_current_ channel?

--
Patrick.