Re: my bison didn't get correct value from flex

Simon Richter <[email protected]> Tue, 5 Jul 2022 16:53:52 +0200
Newsgroups gmane.comp.parsers.bison.general
Message-ID <[email protected]>
Hi,

On 7/5/22 09:55, 徐研辉 wrote:

> I encountered some troubles when attempting to recognize identifiers of C language via flex. As for program shown below, what identifier returned by flex was "main", whereas bison got&nbsp;"main(){
> &nbsp;&nbsp;&nbsp;&nbsp;return&nbsp;3;
> }".

This is a rather common mistake: yytext is a pointer into the input 
buffer, which contains more data beyond the current token. You need to 
look at the yyleng variable to get the length of the current token, and 
you may only look at the text of the token before fetching the next token.

An expensive, but safe, way is to return a copy

     {IDENTIFIER}    yylval.id = strndup(yytext, yyleng); return ID;

This allows rules like

     struct_definition: "struct" ID '{' struct_members '}' ';'

to work properly, because the yytext pointer can become invalid during 
the yylex call that gets the '{' token.

The alternative is something like

     %union {
         struct {
             char const *text;
             unsigned int leng;
         } sv;
         ...
     }
     %token<sv> IDENTIFIER

to get a stringview-like type into yylval, and then use

     {IDENTIFIER}    {
                         yylval.sv.text = yytext;
                         yylval.sv.leng = yyleng;
                         return IDENTIFIER;
                     }

but this needs to be evaluated immediately, either in a mid-rule action, 
like

     goto_statement:
         "goto"
         IDENTIFIER
         <label>{
             $$=find_or_create_label($0.text, $0.leng);
         }
         ';'
     ;

or in a sub-production

     goto_statement: "goto" label ';'

     label: IDENTIFIER
         { $$ = find_or_create_label($1); }

Hope this helps,

    Simon
OpenPGP_0xEBF67A846AABE354.asc (application/pgp-keys, 9.4 KB)
-----BEGIN PGP PUBLIC KEY BLOCK-----

xsFNBEsFLRIBEAC7462Btbl5lLGjG+qgeNHfBUBiZ+q47M1xMC9hAQ5nbCSBqxH7
eSinz4u9DSNp4VT5pTkAcnLj2/f3d4Tis7LVRXA5hWS6aHVKvtk0k7TweXzU5nqG
BvQUuwEzyx/TOQ8rwx+IhjnIgh5g0enhVVNw3DoTXRHtEqQ6twAMj5WccNy9DApF
C07OchJz92/b747juTRL3p0EWQnss8sZ4MKQ7fZhOh6W2QDgvG7ZCKx2UWiRytDz
VY+QgCe6oLHG0kIw7rAgqtosSNGZkPZ8Iav1UVxHDAzOHvyGC48FARvSqExnpJqQ
U+UK5nYacwH4uoL+Su+un/8DMtpfG8m3+DGiX2NgDSfvIXOQczaCLRTr9U7ma5v7
Y+7sWLO7lbIDH6UYovo2C13iTa8qoSIJa1rM6BbwFntAMXtpLIo+DABepHeswy3u
lfb9EmfcS73O/ML+LqlayHUMkwUGx/RsE+ko3asfWmKIMeH9vs7uO2zaZKu/zmIA
Hb6Zi1i2+47kAKIy/gyFeOPkT8vhRMgZlZ7qSKrKl+5jfWwGInGxePabE4TC4nVJ
TZn2Sa0+zb3/h74RLeFpNBhWMH2A0ayIC88mQrzDNSOieaPX+7EjfLBc0n/CUW75
hLe3YlFu0nx/Qit4SyGZP+xN0VYFUw+iVz54+1Io6gQuZHpDwc+nGPBijwARAQAB
zShTaW1vbiBSaWNodGVyIDxTaW1vbi5SaWNodGVyQGhvZ3lyb3MuZGU+wsGnBBMB
CABRAhsBAh4BAheABQsJCAcCBhUKCQgLAgQWAgMBFiEEnEMlNJXk3Kg3lF9b6/Z6
hGqr41QFAmD6ksoYGGhrcHM6Ly9rZXlzLm9wZW5wZ3Aub3JnAAoJEOv2eoRqq+NU
84sQALYTWdkbrpICVH0lqB3SajK7laKosw22BY+GG+y/OiszHcLYiZunv6rAhf5Z
/2QEU4B1U8Ok/nMOvHlTapwVNE7p9aZcJgQWSY7lJsB69SucjN8Vmowlm1bedzil
s9sY0p3CC/HUp0p9SW/vHdZx24yIXc64fky7F9hGe3nOYOuT0gbTIm3qKK/AWQp/
gSpD5vRanb/6ClG+jFQQK4lNogyLPiLZRJFCFSDlkI89PiPA0Z1bAoWexTgF3v6L
LSCYjx0PUQZrmARHULqCh97dTMGeveoiKO+wDMUk1T+TzLM0gfjbKvP8yEdkHxTB
M0xo/gzChsTXxlBqUnP1h4IeadXBBEL6CLu8VD9hVVBOYsd1nudJ0xHXD7gtSzTl
RlbDE0cpS4OA7fNdkuFNJ14Peo+SMc46lWYP3eOI8j75QOW29yW7UZfQVRGkAwwG
ZplFhEHm1hxPXUF6BKALgoaok+Xyu0hBKykg4J3FjxQnWJE2UHv+et/2lSg+/dym
fDY2vHd7V3dsuDD1g2iBQx4G/MQcym4HFGSTHyNePEiZkDJAtXT0YiuwrltSTJU6
cmHwpOGMV3qQEwnZabKiUB4G1cakZANGkvuWFEVXXBiqGFvw0Okb3NhdHvOK9WbO
zMsRg9dDEnzgsR2kM/PXUegdSc7cro22lUk/Gt0pzURbfyXFwsGOBBMBCAAhBQJL
BS3AAhsBBQsJCAcDBRUKCQgLBRYCAwEAAh4BAheAACEJEOv2eoRqq+NUFiEEnEMl
NJXk3Kg3lF9b6/Z6hGqr41RpMQ//b8klHzCrrSg33lknh0QxDMatQGj/ptdrJw0k
polKNXNUYTa9j8SXwFdFVUzTtQoTgSUf8hLdYAgOsH8IazNPqSoLAU8OWjaez6Cj
SfHhYiJHcZqKBmr2RFUAfLlfk23+7JL7W5UBtVA7qGuLIn2uL3D/H8i6Hvhoq1Tp
TCN2Sw02phmdTzwt1w3NAVdm6nGj6sBB8OgQbKHbUTZCx3W8sPGRN38SKkQuIMdD
zJMPSHLzi2ZnP+ThuZ8+htlLQ3LHCRIreKINleaCW9gOHqMGrTtCitN/ia8ViG7l
oEvkgJmLLIGW0obk5moWMmU0Dg8Y4dMcLMVqNjqP9JAQDuv4lCxPHKBIVyFpUqil
MvhGCPLbt19Y1HkCWa/WfYXmotCwlhjOX7AGPUBRdMpo95ajuEqXyK0vUcIoUCml
3uU1r3sGYL//TiBLpP/9Wq1KMTC1CskBkrRp/FT28vzAZ2ZkATWv0H+yaf5f6zz4
IhlEoP/Htr8+wc/dVaSA0JdpE3wQOCH2VycMJnmZvfHu8POaroZrGnEBYLugcuJd
W/aHI73M9On8Q+2P4fuIqbF/OVdSFNrWTLVGmL+tNhNH3R7SUZyH+07AZbAjuSGj
jRSosSloEg3jn67AoqBWpfrDVprtt/ozMI31fod5Jylr2xl3cWz+mdyMPTGUl+p7
Ork2VWnNHlNpbW9uIFJpY2h0ZXIgPHNqckBkZWJpYW4ub3JnPsLBpwQTAQgAUQIb
AQIeAQIXgAULCQgHAgYVCgkICwIEFgIDARYhBJxDJTSV5NyoN5RfW+v2eoRqq+NU
BQJg+pLJGBhoa3BzOi8va2V5cy5vcGVucGdwLm9yZwAKCRDr9nqEaqvjVJaJEAC0
/w0rEaK0LlRPLdIsZSpoGKALzyGXWctYRWSQb9JcmXnbxJhZ2ElGHZFZwghT80uz
NTKsOchdSFrYkAEcjBtBaD8ERELbqj/uRSFHmpEkGQ5OnLry/Yoq2M0UbNX6dOeH
tPDFro52aP0+NxkLm0UxFRc++d3hKBoBei5NOLgzeFEMK6nyQp+v9KD6HMpHxgaP
jKt3WPbtHh5MbV8TMeEaCBwzH+os7VmTjsOs3A4aNNWbpATFb0bNM6gpAvvnp5G9
AEJMFEKGjPhNAvYJHxHx7IlqV6k403RWNxFcugjkEWtqowzD1R7+921nw/0PG7eM
80TiYFot5SseuPmAyDTMjSVfbcMp9lrvlBpwC7T6vKUPO2qXEMbFSl+iDuSRsIIL
st32ItyHg47WBrBfIiTyaHXjvkfvCJgesZMIspfD44gR4iPKAeInXXVzwZC2ai4t
b81iaS6yYWVHfmwLfiHwj3HKP1NqmbipIgTcxvRtgaiE3CaT73dtPUjGONMUHCqK
O6eNqoVAYVApgFh4XK4ZIHdoFHtF6HUW8v+N5jQkYjT7g99oKvD07Sw6BdhOhuHe
wfjW8qWTL85IdJjPsJtkWNGsiDe9zrA7LT7/unJEGj8Ulv5jg/p4FTqP+corCwZg
x1HX/8m+qqFiYyMma3k2tTw1le7eKW5l3VP4S65iScLBjgQTAQgAIQUCSwUt2QIb
AQULCQgHAwUVCgkICwUWAgMBAAIeAQIXgAAhCRDr9nqEaqvjVBYhBJxDJTSV5Nyo
N5RfW+v2eoRqq+NUGv8QAKn1Bi8qAa/BRtDXjmS/j5sLopQ7htUWNOkY30MsNGCk
XSzwY4Zl3mSHswNrFrtj2CqU7+gJYWDe1PVvmbBwkJOBWrnApIS/JN5bER9fMojR
2FBR12h5C0oxYLhraCc0tySlLA4rcoH7OI4ctou86B4idnVx03N1zZ9tH6vnk000
6ih4iH+DiP0lqyYbLquuwoUTj0FnryWvb08pHGlHmrF+r98WnhMrOOTLr2KV2U4X
Ta6xqMqC+KRyKgQcUHY/5oicHci/7FzfolfI9P3kSaoczj6y+dbE88SyiqpTUr9j
TlI+C+zNdtwbBSHssYZrgaZ2R9mAMSPbjCnwSseBniRi7E0SJRhyWRMgwNVr7XfS
Vbq8kDKzJAdoRExaakVxCvVmeu78w/mdyIclMDXuryZHX/if3/HlMcredoEfJ/I8
QNwJY5Ush5Tw0BqOoLfWwVHcQ9F0Nx1ThPpF6GwgaFa0s7AnBQVhNZyPWmFS9QP2
6mnAwdNgfk70JBfMyBZ0Stn51pLGh/oQOLpuTem+41lBh8dTL8geBSh25ubqXk3S
dY1gaOeTxnxZ082JC+ReEzRgAM4vBkbdZa4fHN3AxnuS7sI9ybadwk37lv+/+/5U
gPuJDEziESaplfXVTkD8qIyq+lUEnG7kUiMyx2TImWqrWxcIjeg3pr29zCw5XcGf
zSJTaW1vbiBSaWNodGVyIDxzckBzaW1vbnJpY2h0ZXIuZXU+wsGqBBMBCgBUAhsB
Ah4BAheABQsJCAcCBhUKCQgLAgQWAgMBAhkBFiEEnEMlNJXk3Kg3lF9b6/Z6hGqr
41QFAmD6kskYGGhrcHM6Ly9rZXlzLm9wZW5wZ3Aub3JnAAoJEOv2eoRqq+NU4g4P
/A42zMOoMnPB3AK7VRNLFgVesi4OSpl5KAlNXQXVwjiMKNHFssWXOdS4GIQZywp/
STk2i456ldvWDEtxpP+2VKhaYd3gEVNw9AsFOx1UPRC5eHycUJidStDT8xs4Nyec
b9t489pcH3gWGPCsasBucQ+bJhrUaLjcroW4ebkMYGlZo2tjLnwo/q0WTJsEghAI
IyMZXtS2wF9btEabRtuW0XuHDoikg8ZNApF8hJ5KGZpt2q0hvORsMvuOxTDpIV7Y
LfFImT3Xhkzns5pAOq0CeaCJiNoAsgCJmZw9KsC43+mtUbG7MiUuvBhSljdFrynv
r5Kt5Axtp1sg4w8/82R0RoB53Z0et0YXP2ofFQ8TIm6Qmw9Qzd6P5CObrNMyo8sU
nap4HFqeDkZRG9xmmtkcY9/y+r3PaId8o/DPdFr3EBugYQfpo/GyXeacZ4ZnOGtt
Sv194Gy66EftVbEhVXfmX15o/lGu5vrC9zMFzBznT+agBAm3t3fF83nVonGaQjkO
/DjlTI3g1uLVY5xRX/DImNF5b2g3Iu9zVwHbJDcMDpgE6TQfyagMxBpDsyx9ZyhR
eGW6AHsvZ+Xda9C+t2IvwB4/uuJMm/PNnTSu2Jht5lCjSrQCwNfYcku/W4wfBbFe
lndII4KDng/iVQDnGEx2fuCtvpnSlqVtPEf8e7aPsq4EwsGPBBMBAgAiBQJVkaFP
AhsBBgsJCAcDAgYVCAIJCgsEFgIDAQIeAQIXgAAhCRDr9nqEaqvjVBYhBJxDJTSV
5NyoN5RfW+v2eoRqq+NUIDAP/R0GnDw65HCfrQMtUbWupbJKH2P2AsqIGHMujPQF
77eMUWetX0xsb1V9rF2viPFuQzzuZ6iORrmhGilnYleTrxkjeLi77va6eGrLxjEn
pQctdsKCtfqGvsW92takDyI9Oyi8f01SJHqHtByWB8teEHtWwudMEKzsqGsYkK/g
rfPFMZZ36WiE8nglL3Hitcu2YWS1ftBpQQc+kPkdU4h+CFcaxAVr5oPCW43VKFyj
Lyo7rafD5k3UgsFlm1ob3ugcwqp3z++zoxoViJNlDGfzw1P+/hYHWAKtJ0kohrlx
QOXGlWg/aJAsBEvukU0iP4VqaeE4dMqAf/jv0iMHNM0LwY5VSVrfVDEejE/D8rrZ
WJABL2v3ee1BLD14ujnOyfJi5qzKE5mOmMtjtfMEpsIj1+ALvzQ5NnP0OcKcCfeo
ACJ+A/NKx3l16s22I8Dod6SrmQDKYC577ixaw+g5RpftPE8it3YUsOAv4sWgQw9D
yMPbVFRkc8m62KuCRygCrok7zzA9DAbOpy2sHICTH7phK1s0odWcv2bOcgKnLiN/
U0tVmiAIOKTMm576efMvhcVr38QKmLG48KhxjyzNWUWOFVqhy8cUPRD1LNCcOanQ
/oX5f4WecLLOQIP2ksM9GjpMx07iwXY3skzffpuaCtOxzjBsoSKhruSbsozrNgBP
uoU0zsBNBEsFMIABCACVJGrpCMiJake4CADp6DOESYYvUJQHkDVMpOax+/qNhHiQ
ckDYj7CGI7/ZVWZOS22SFhFF0RmZidXhHrV/8nhC8eWnzdg66+2b1W0u3uXRAjEm
hI2YPXvLXJupoqTk0GhGFPVaf+uTIaf/cOvCTgrH+H4h4zK0VDwgQ2OFjEozoYdV
2AeCV+cjfg+NUSbrEi5hDMN+9xjhQ8hKpKnTkZlHrrLYs/YmqTaHqUnVPJyVe+In
S9bKFO6QcMiV36Sla+dninRLXaY9GzGQfZOytgjqQhduFY8uEwGwJTLSjdA8C8xu
R3pMFDcV335j7alwKmYHdF0NgRKiUS5UNNgtnVjBABEBAAHCwXYEGAECAAkFAksF
MIACGyAAIQkQ6/Z6hGqr41QWIQScQyU0leTcqDeUX1vr9nqEaqvjVGwWD/4mcV/w
SWRCwOatQ+Q8qwZ94KMnDMpiiSz/Pu84E7eyuV4WFtslUZqYI2KaLSYlu4Fj1RjZ
uPCbiIGvSl5dvdqVJJEtuGASJT2N1+DZVLGlM0BMNL/hT6zzOq9+hkstETQ6yyxF
8boquPrpzCpKP2Pv3MzYQL420onkNvTONIhYRhwFW/UgILxJkHY8FKXdfehemZeI
rSA8iFTxXcirzJ9xaJm4Hn33oHy9psRjSOP+JxzgQ2w7dZHc/SJiogy7tDSkrVWz
jh8Kcq4PInqwTrIR128tq5gUW26oczP2Tu6YeljcT4g6kKjXMqZ/ju8WDQtTHv44
b3lvg7VCrUul/LMzkeOIOinAgXn9VDELwr6Tw3XgZmP9d8Qe9+xl8Ivb7pYGioy6
ZgqtbcbXtFSKjnrbAZQzp+9A0jfPOqZLmcWPeC84z2L9huE5WxhNpt1woBTFg/t3
PGqn6hy3bdwZWp3UL51q1rQ1H/sUgSyV+6ubFc+p5vLyGEX/0j/+S9FWDME8cUPP
FtHZRg6N7Oxd75fkR24d824mnVet8wHl9A/rKc76TygVPZgS3H8n4YI/BgsDK29p
huRJp+IPIJBZoMdPLyxXP/QFNsfKOt+0Bn7+J36S3dSTmhxckkt1fJtoVYc+xwwP
v/+QQQny14d9pq/rJa4KTG64EctGElv9baTANc7ATQRLBTBCAQgAuzB7uKcwHgn2
EW1juT/2FRzgcBV4E6Re139ATiiMkSCrrZhaZZPeTXOkMW57Oaia0ZoSaMGknUAN
n6jJf/SnJhPKb8SWwXQuWQuXTywsro5z2ZjPToWLzwvrJ8x8CaEAoqahiuT96SRU
ke7IWBi8HciGoCCIrNLMd+XoFc9OzZlGTtxVVp1z2ba/f2lmVKHk4xvigMMY81e5
6dFsCVE7Gu3ubu6qm1bxe1RSn/MKzzIgRmy1BeJ2iThieOQZFTpW62o8nC2eVbJU
BF4If8TibYgJTbjepEEIq494Ix7OXSev0RYtbCxBiUae/8dKn6RYOtQhxa/++vbo
kMdEaIzqkwARAQABwsKVBBgBAgAJBQJLBTBCAhsCAUAJEOv2eoRqq+NUwF0gBBkB
AgAGBQJLBTBCAAoJEH69OHuwmQgRy9wH/3VdaFVqM93rhOeMi03gl28sW16VIfR2
pvGz5JFgYuxhxKPZzSumSwyUwIVRI43pD+xo+GrLiMIMi/hYcB3FiLEEFvV5rt/1
Y3nrfDr+qGgIRGYATOduSnSqqPH+c7sTxbIS7DHK9HYiL1J1QvD7ORheL3f8iIe/
cvjNYtbKH4wHeh64AEN1xDEAsMPL6x7+cCcZKfKvFQTWtbtae/5GHEeP0vdA46zg
OT7ShXWFBNlsE2NVEuk6ijCXcA7wT/xg9dYQeA8e8m/q6D+z6u19UAXuITn8kgBR
HIiMR+xN1a+O1SP1VkL5M46U8JZQ3uawsmAfKtta/bbvcWXYA2PwBZQWIQScQyU0
leTcqDeUX1vr9nqEaqvjVMgrEACDT36wK3VdpPU8gbPom9zUAKlQE94LzwcrXuiV
qto38JhErMiS/RGUFoEbFgf7/NUo2e8AL+IZExySHv++hTVmuSu8PO3kh+shXOaH
ViYCPIC0BFYA3da3fCtbdCmKkbmh7SUV+wb7naYx0vReLe0u90OmIHYq5pKHCWvk
t1m42ux8KKSEWogL8/zeKjKEpiO0KTECVUR0sgKy8Svl5mhgz1rNdXOYKjiZ/6UA
7f/zlZfCOAkLEbl/FI0bF12k138mdBp/gM6DPv3M++EEnkuQmcBOylH3e5R6/Pnj
jmhfEWBTsX5aW6UcJbrf/RyCbZn2tJnge185iERGM4LNcLZbJdYQYUG2DmtRG0N0
T6227/179JWlR3oOtyMPxNyXzsRnU1CzdFA+T/63xmRRheriPfezdF68QH1JXl6J
Ixh0Na/IIalakRqHOJuUgPhZGnboicD83bbBTwZdid+xAlTwgUOTNbZKdSaDWp2d
WH6nK4RDBhR9ywryHkHzfetBpdqqrWsUv+pCyyf04iqRIbOoLSz2sFU8X3xX/1V4
sWevky3VKbhuEZuEtIfbekSc0rnUoXSPiALCHoufH6OqDDJgyPrL4kRmQbHIOX4y
AcvoOjNUWLOaiU+tADuDMp5kxZ+Atld+IuPqaziV1kFXKcILiB7qJkyxk9Ug4UhP
qHpXys7ATQRLBS8NAQgAzVHagDTkE5blcanb9uiZWgHqh1V+u11f7HSGDRe0j9sx
Y+koAcKmH+I4IEV+jABXFTlpuuJstAKKfC4llhioibjSm4W2sAS7f/5lnHRcQ4uy
c7dfo4F9yqNJVmWSHo5yGelTf9bb2VWX9fpaytRCNZVD14wOPq6HftqeaFSz0IYS
Sth+xOzr8P5YPzAN7CpKJRB98iYs8fVL8qsFp/lNhbI90hYvboUaFYKG1sqD9t/8
pG63IsVL4ZsY4+zFVrwD2JvcjYzlGStoFiaJfvN02I8rXoDq7k8QFtRYlCQaLPqf
cUuNPUXVIWqm8Zwka7AtOizEIFpLTNUTSiDs+Cfu3QARAQABwsF2BBgBAgAJBQJL
BS8NAhsMACEJEOv2eoRqq+NUFiEEnEMlNJXk3Kg3lF9b6/Z6hGqr41QFmw//fDxL
BoXMsAd48JCJfqCX88BTQ0rJl+P2uvdKXebCpXslGCRxzq80VM5lRdqxmnwjv5CA
4TqsP0zakN1MWeYNF0nAQd8nXGOn9zxPGM5GiEU5WuEQDwDVKAzN8bFlBnHD/c8r
QA8xh6FpZaQQmtFeEtcHU4ZXF38QH+aU9K11VljFa4yzF4Ctcs+cmiQSWMl/DHHF
tFkKoeb1dDvJLk9GJlBBXlcu6+R1R33ORjgTVeMEjXYpM2iycMKwby4P79yTVFKt
pterKaUorucnYgZMboGYicAy3scxjhq7qlSZNwcY/poawIlIdKA0mwXGTuqd4Jhu
SJ22JeGz9cwjEuzeUyNNsZ6ARrA4vQVHDQDMUupo8Ia7evsFV7+tr4wsaeV5be72
LlJOM4nf5qMSqAdSW/JRkxO+rz+ZDmacWRMJW2/59BUvfDbOZZbwj4CobYofCDB2
YXNrkOMBoof0HOafhHdsQKzuabWZ5caFBCTST20zzn623Qr4aPI7kwKN+rk8GE3W
7tS4phxUdTi13k6EjQHX34VPJ5T7giIDZaNiuDSRZlP7Z9Rd80yjJJJjBfqvgYoX
896WlqRMn25d0x+PHgc8gDYvOT76blqh0HBA33G0mMkgYlsgQCl2bA1Sl0u1fWSY
99zQVgeYoaLMy3HgfCAph91x70zKkmHV7kVxqqQ=
=l1MO
-----END PGP PUBLIC KEY BLOCK-----
OpenPGP_signature (application/pgp-signature, 488 B) - not displayed