magic and length

--Sig_/rIlANk3nOc+mftVOlwsZ+76
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

Deep inside the bowels of some XS code, I see a SV like this:

SV =3D PVMG(0x23ee760) at 0x23b10b8
  REFCNT =3D 2
  FLAGS =3D (SMG,POK,pPOK,UTF8)
  IV =3D 0
  NV =3D 0
  PV =3D 0x2456e20 ""\0 [UTF8 ""]
  CUR =3D 0
  LEN =3D 10
  MAGIC =3D 0x224d650
    MG_VIRTUAL =3D &PL_vtbl_utf8
    MG_TYPE =3D PERL_MAGIC_utf8(w)
    MG_LEN =3D 7

If I - in pure perl - ask its length using

  say length ($sv)

it returns me 7, where I obviously expect 0

The sv_dump was called immediately after a cleanup call:

  sv_setpvn (sv, "", 0);
  sv_dump (sv);

Does that code miss something to reset the magic "length"?
Is something not cleared in sv_setpvn?

This is perl 5, version 24, subversion 0 (v5.24.0) built for x86_64-linux-t=
hread-multi-ld

--=20
H.Merijn Brand  http://tux.nl   Perl Monger  http://amsterdam.pm.org/
using perl5.00307 .. 5.25   porting perl5 on HP-UX, AIX, and openSUSE
http://mirrors.develooper.com/hpux/        http://www.test-smoke.org/
http://qa.perl.org   http://www.goldmark.org/jeff/stupid-disclaimers/

--Sig_/rIlANk3nOc+mftVOlwsZ+76
Content-Type: application/pgp-signature
Content-Description: OpenPGP digital signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBAgAGBQJYz45lAAoJEAOhR6E+XcCYiJwIALjb2kfKAzdBzwQBJ1Gb7M6r
OaIq1y9hsMFnpGBZFuDjRAG3eTssuZoqZAJd2jFqxEEwRUQDdx6QMFdJ9aJoWH95
70wvP+Mm7xKDgE11JWBZPZ4cQP28d0YqZxM04Y9xdfk2cq03xydzY/jRqi24xVIR
78cel7H4WTeEG2XjJQU977uI19wNlKoj8X2xsOMT4o6NnRW6WiYRzZF+1FSdnvc0
dV/1d0Cnr+s7/bRkm+1UtsI/ZbsOuevyn5ptvUKFvsx+eQ7/OoM5zgdAeiPXHzxj
6UoFQJjIjMDQd9sxek1n2Ie0TMQ6THAHiv1L3gOib4YUS2gpginMhrTZjOOMDU0=
=E8o/
-----END PGP SIGNATURE-----

--Sig_/rIlANk3nOc+mftVOlwsZ+76--
0
h
3/20/2017 8:10:05 AM
perl.perl5.porters 46379 articles. 0 followers. Follow

3 Replies
34 Views

Similar Articles

[PageSpeed] 26

--Sig_/UYsO8MnR9VJs.wW3QlGpnQF
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

On Mon, 20 Mar 2017 09:10:05 +0100, "H.Merijn Brand"
<h.m.brand@xs4all.nl> wrote:

> Deep inside the bowels of some XS code, I see a SV like this:
>=20
> SV =3D PVMG(0x23ee760) at 0x23b10b8
>   REFCNT =3D 2
>   FLAGS =3D (SMG,POK,pPOK,UTF8)
>   IV =3D 0
>   NV =3D 0
>   PV =3D 0x2456e20 ""\0 [UTF8 ""]
>   CUR =3D 0
>   LEN =3D 10
>   MAGIC =3D 0x224d650
>     MG_VIRTUAL =3D &PL_vtbl_utf8
>     MG_TYPE =3D PERL_MAGIC_utf8(w)
>     MG_LEN =3D 7
>=20
> If I - in pure perl - ask its length using
>=20
>   say length ($sv)
>=20
> it returns me 7, where I obviously expect 0
>=20
> The sv_dump was called immediately after a cleanup call:
>=20
>   sv_setpvn (sv, "", 0);
>   sv_dump (sv);
>=20
> Does that code miss something to reset the magic "length"?
> Is something not cleared in sv_setpvn?

If I change that code to

  sv_setpvn (sv, "", 0);
  if (SvUTF8 (sv)) SvUTF8_off (sv);
  sv_dump (sv);

SV =3D PVMG(0x17ca760) at 0x178c518
  REFCNT =3D 2
  FLAGS =3D (SMG,POK,pPOK)
  IV =3D 0
  NV =3D 0
  PV =3D 0x1832e20 ""\0
  CUR =3D 0
  LEN =3D 10
  MAGIC =3D 0x1629650
    MG_VIRTUAL =3D &PL_vtbl_utf8
    MG_TYPE =3D PERL_MAGIC_utf8(w)
    MG_LEN =3D 7

and length is 0, as expected. That however does not look like it is
supposed to be.

> This is perl 5, version 24, subversion 0 (v5.24.0) built for x86_64-linux=
-thread-multi-ld

--=20
H.Merijn Brand  http://tux.nl   Perl Monger  http://amsterdam.pm.org/
using perl5.00307 .. 5.25   porting perl5 on HP-UX, AIX, and openSUSE
http://mirrors.develooper.com/hpux/        http://www.test-smoke.org/
http://qa.perl.org   http://www.goldmark.org/jeff/stupid-disclaimers/

--Sig_/UYsO8MnR9VJs.wW3QlGpnQF
Content-Type: application/pgp-signature
Content-Description: OpenPGP digital signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBAgAGBQJYz5BKAAoJEAOhR6E+XcCYRSEH/RmWskF4MLiYyzPgFwhVgEnI
zbZt+rI+kP3bVOs44otTYIoPyIG6vJxd0VEF/+h3cwqqgGsCWe/9boUBCXhnn43Q
8zRW1IBx+32HCHeTg9MaUNpuDYcCjh/ll5Od1DF/BmjiKrccy9Zz/PtEhZaBeMEQ
LU/Vys0cdAjP9eY+zHjlDWZClsd/CzYzPTB7DZc7VpNDo5+iCxFISDY2zAB2P65V
CvsTftUyTlIBzG0F9F+821QbanoZZPoP3g5yugW0sEQnH32ERTCG/Kqiv49b04Ux
l8FZd/maj7dKF6j0Ac/aF2DItcpZNpS7BOGWzGnedk/wqpSnqcwj0XECgtEsvLo=
=aq2G
-----END PGP SIGNATURE-----

--Sig_/UYsO8MnR9VJs.wW3QlGpnQF--
0
h
3/20/2017 8:18:11 AM
On Mon, Mar 20, 2017 at 09:18:11AM +0100, H.Merijn Brand wrote:
> On Mon, 20 Mar 2017 09:10:05 +0100, "H.Merijn Brand"
> <h.m.brand@xs4all.nl> wrote:
> 
> > Deep inside the bowels of some XS code, I see a SV like this:
> > 
> > SV = PVMG(0x23ee760) at 0x23b10b8
> >   REFCNT = 2
> >   FLAGS = (SMG,POK,pPOK,UTF8)
> >   IV = 0
> >   NV = 0
> >   PV = 0x2456e20 ""\0 [UTF8 ""]
> >   CUR = 0
> >   LEN = 10
> >   MAGIC = 0x224d650
> >     MG_VIRTUAL = &PL_vtbl_utf8
> >     MG_TYPE = PERL_MAGIC_utf8(w)
> >     MG_LEN = 7
> > 
> > If I - in pure perl - ask its length using
> > 
> >   say length ($sv)
> > 
> > it returns me 7, where I obviously expect 0
> > 
> > The sv_dump was called immediately after a cleanup call:
> > 
> >   sv_setpvn (sv, "", 0);
> >   sv_dump (sv);
> > 
> > Does that code miss something to reset the magic "length"?
> > Is something not cleared in sv_setpvn?
> 
> If I change that code to
> 
>   sv_setpvn (sv, "", 0);
>   if (SvUTF8 (sv)) SvUTF8_off (sv);
>   sv_dump (sv);
> 
> SV = PVMG(0x17ca760) at 0x178c518
>   REFCNT = 2
>   FLAGS = (SMG,POK,pPOK)
>   IV = 0
>   NV = 0
>   PV = 0x1832e20 ""\0
>   CUR = 0
>   LEN = 10
>   MAGIC = 0x1629650
>     MG_VIRTUAL = &PL_vtbl_utf8
>     MG_TYPE = PERL_MAGIC_utf8(w)
>     MG_LEN = 7
> 
> and length is 0, as expected. That however does not look like it is
> supposed to be.

The attached magic caches byte <-> utf8 conversions. IIRC, it caches a
length mapping plus up to two mid-string mappings - useful for things like
pos().

If they are getting out of sync, then either there's a bug in perl
somewhere, or you're not calling set magic after changing the value of the
SV.. Given that sv_setpvn is documented as "Does not handle 'set' magic",
it looks like you need to use sv_setpvn_mg() instead.

-- 
A power surge on the Bridge is rapidly and correctly diagnosed as a faulty
capacitor by the highly-trained and competent engineering staff.
    -- Things That Never Happen in "Star Trek" #9
0
davem
3/20/2017 9:19:03 AM
--Sig_/Iy2Tfz3BTXZ3TRal.+mD/fl
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

On Mon, 20 Mar 2017 09:19:03 +0000, Dave Mitchell <davem@iabyn.com>
wrote:

> On Mon, Mar 20, 2017 at 09:18:11AM +0100, H.Merijn Brand wrote:
> > On Mon, 20 Mar 2017 09:10:05 +0100, "H.Merijn Brand"
> > <h.m.brand@xs4all.nl> wrote:
> >  =20
> > > Deep inside the bowels of some XS code, I see a SV like this:
> > >=20
> > > SV =3D PVMG(0x23ee760) at 0x23b10b8
> > >   REFCNT =3D 2
> > >   FLAGS =3D (SMG,POK,pPOK,UTF8)
> > >   IV =3D 0
> > >   NV =3D 0
> > >   PV =3D 0x2456e20 ""\0 [UTF8 ""]
> > >   CUR =3D 0
> > >   LEN =3D 10
> > >   MAGIC =3D 0x224d650
> > >     MG_VIRTUAL =3D &PL_vtbl_utf8
> > >     MG_TYPE =3D PERL_MAGIC_utf8(w)
> > >     MG_LEN =3D 7
> > >=20
> > > If I - in pure perl - ask its length using
> > >=20
> > >   say length ($sv)
> > >=20
> > > it returns me 7, where I obviously expect 0
> > >=20
> > > The sv_dump was called immediately after a cleanup call:
> > >=20
> > >   sv_setpvn (sv, "", 0);
> > >   sv_dump (sv);
> > >=20
> > > Does that code miss something to reset the magic "length"?
> > > Is something not cleared in sv_setpvn? =20
> >=20
> > If I change that code to
> >=20
> >   sv_setpvn (sv, "", 0);
> >   if (SvUTF8 (sv)) SvUTF8_off (sv);
> >   sv_dump (sv);
> >=20
> > SV =3D PVMG(0x17ca760) at 0x178c518
> >   REFCNT =3D 2
> >   FLAGS =3D (SMG,POK,pPOK)
> >   IV =3D 0
> >   NV =3D 0
> >   PV =3D 0x1832e20 ""\0
> >   CUR =3D 0
> >   LEN =3D 10
> >   MAGIC =3D 0x1629650
> >     MG_VIRTUAL =3D &PL_vtbl_utf8
> >     MG_TYPE =3D PERL_MAGIC_utf8(w)
> >     MG_LEN =3D 7
> >=20
> > and length is 0, as expected. That however does not look like it is
> > supposed to be. =20
>=20
> The attached magic caches byte <-> utf8 conversions. IIRC, it caches a
> length mapping plus up to two mid-string mappings - useful for things like
> pos().
>=20
> If they are getting out of sync, then either there's a bug in perl

If so, we need more tests :)

> somewhere, or you're not calling set magic after changing the value of the
> SV.. Given that sv_setpvn is documented as "Does not handle 'set' magic",
> it looks like you need to use sv_setpvn_mg() instead.

Spot-on. Thanks!

--=20
H.Merijn Brand  http://tux.nl   Perl Monger  http://amsterdam.pm.org/
using perl5.00307 .. 5.25   porting perl5 on HP-UX, AIX, and openSUSE
http://mirrors.develooper.com/hpux/        http://www.test-smoke.org/
http://qa.perl.org   http://www.goldmark.org/jeff/stupid-disclaimers/

--Sig_/Iy2Tfz3BTXZ3TRal.+mD/fl
Content-Type: application/pgp-signature
Content-Description: OpenPGP digital signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBAgAGBQJYz7OAAAoJEAOhR6E+XcCYzN8H+wW2ElODMDsl7hcDNmHQ8E5p
yP2yPzyE0SzLAtHKo+1bM3c54rL1cIgMbkUokN6dfZMzg1048XAu/FUjkCP6zS1f
OOplQubtzahRsEth9TtmqVp9qrBsvV7ytfXoRtsTzSE5aeOr4LKY2yhnJ9Rn4ODO
ERjBeHbH0ynOhF082si89vPjHAW582sBSIJfcj1UE9J9pAo6zot6hbM2fRxn2Iby
A/YHCoE2DSxEWvQivRD7AzxEHKWQRxypsMFbLNO9zUYfy7aFvxhINiGZstFH+Ad7
i5BxlxRa2f5sYAktuu49FpgnvHUhP6IGCMZtO8a2pSvPOo77jr/CpkRNa/BfNGk=
=Y/Av
-----END PGP SIGNATURE-----

--Sig_/Iy2Tfz3BTXZ3TRal.+mD/fl--
0
h
3/20/2017 10:48:27 AM
Reply: