Re: Question: Can the lexer access the token value?

This is a multi-part message in MIME format.
--===============3922784831741248047==
Content-Type: multipart/alternative;
 boundary="------------2EF49C2590A42D896AE41ADD"
Content-Language: en-GB

This is a multi-part message in MIME format.
--------------2EF49C2590A42D896AE41ADD
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Op 14-06-2021 om 00:10 schreef brian moroz:
> Hi,
>
> Question: Can the lexer access/retrieve the actual string token during
> scanning. For example, Flex can do this using the yytext() function.
> The examples I've seen on the web including the Boost site do not
> illustrate how this can be done using just Spirit lex.

Yes, you need to pass lexer `std::string` in the type list for the token
type e.g.

|typedef lex::lexertl::token<char const*, boost::mpl::vector<lex::omit,
std::string>, boost::mpl::true_> token_type; |

I'm pretty sure I have several answers showing such lexers on
Stackoverflow: https://stackoverflow.com/search?q=lex+token+omit (those
keywords invariably appear in this kind of declaration).

I hope you will find things here that either answer your question or
help you make it more specific,

Regards,

Seth

A propos of nothing: I could have a look at the actual work. I usually
recommend going without Lex

>
> Thanks,
> Brian
>
> -- 
> *Brian E. Moroz*
> /[email protected] <mailto:[email protected]>/
> /[email protected] <mailto:[email protected]>/
>
>
> _______________________________________________
> Spirit-general mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/spirit-general



--------------2EF49C2590A42D896AE41ADD
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: base64

PGh0bWw+CiAgPGhlYWQ+CiAgICA8bWV0YSBodHRwLWVxdWl2PSJDb250ZW50LVR5cGUiIGNv
bnRlbnQ9InRleHQvaHRtbDsgY2hhcnNldD1VVEYtOCI+CiAgPC9oZWFkPgogIDxib2R5Pgog
ICAgPGRpdiBjbGFzcz0ibW96LWNpdGUtcHJlZml4Ij5PcCAxNC0wNi0yMDIxIG9tIDAwOjEw
IHNjaHJlZWYgYnJpYW4KICAgICAgbW9yb3o6PGJyPgogICAgPC9kaXY+CiAgICA8YmxvY2tx
dW90ZSB0eXBlPSJjaXRlIgpjaXRlPSJtaWQ6Q0FLPXFtazczVi1yY2YwQm9ISGstR3k3V3ZI
TjF6ZzJDMENnSjVNWG53MHhGa2pMPVBBQG1haWwuZ21haWwuY29tIj4KICAgICAgPGRpdiBk
aXI9Imx0ciI+CiAgICAgICAgPGRpdj5IaSw8L2Rpdj4KICAgICAgICA8ZGl2Pjxicj4KICAg
ICAgICA8L2Rpdj4KICAgICAgICA8ZGl2PlF1ZXN0aW9uOiBDYW4gdGhlIGxleGVyIGFjY2Vz
cy9yZXRyaWV2ZSB0aGUgYWN0dWFsIHN0cmluZwogICAgICAgICAgdG9rZW4gZHVyaW5nIHNj
YW5uaW5nLiBGb3IgZXhhbXBsZSwgRmxleCBjYW4gZG8gdGhpcyB1c2luZyB0aGUKICAgICAg
ICAgIHl5dGV4dCgpIGZ1bmN0aW9uLiBUaGUgZXhhbXBsZXMgSSd2ZSBzZWVuIG9uIHRoZSB3
ZWIgaW5jbHVkaW5nCiAgICAgICAgICB0aGUgQm9vc3Qgc2l0ZSBkbyBub3QgaWxsdXN0cmF0
ZSBob3cgdGhpcyBjYW4gYmUgZG9uZSB1c2luZwogICAgICAgICAganVzdCBTcGlyaXQgbGV4
LjwvZGl2PgogICAgICA8L2Rpdj4KICAgIDwvYmxvY2txdW90ZT4KICAgIDxwPlllcywgeW91
IG5lZWQgdG8gcGFzcyBsZXhlciBgc3RkOjpzdHJpbmdgIGluIHRoZSB0eXBlIGxpc3QgZm9y
CiAgICAgIHRoZSB0b2tlbiB0eXBlIGUuZy48L3A+CiAgICA8cHJlIGNsYXNzPSJsYW5nLWNw
cCBzLWNvZGUtYmxvY2sgaGxqcyIgc3R5bGU9Im1hcmdpbi10b3A6IDBweDsgbWFyZ2luLXJp
Z2h0OiAwcHg7IG1hcmdpbi1ib3R0b206IGNhbGModmFyKC0tcy1wcm9zZS1zcGFjaW5nKSAr
IDAuNGVtKTsgbWFyZ2luLWxlZnQ6IDBweDsgcGFkZGluZzogMTJweDsgYm9yZGVyOiAwcHg7
IGZvbnQtc3R5bGU6IG5vcm1hbDsgZm9udC12YXJpYW50LWxpZ2F0dXJlczogbm9ybWFsOyBm
b250LXZhcmlhbnQtY2Fwczogbm9ybWFsOyBmb250LXZhcmlhbnQtbnVtZXJpYzogaW5oZXJp
dDsgZm9udC12YXJpYW50LWVhc3QtYXNpYW46IGluaGVyaXQ7IGZvbnQtd2VpZ2h0OiA0MDA7
IGZvbnQtc3RyZXRjaDogaW5oZXJpdDsgbGluZS1oZWlnaHQ6IDEuMzA3Njk7IGZvbnQtZmFt
aWx5OiB2YXIoLS1mZi1tb25vKTsgZm9udC1zaXplOiAxM3B4OyB2ZXJ0aWNhbC1hbGlnbjog
YmFzZWxpbmU7IGJveC1zaXppbmc6IGluaGVyaXQ7IHdpZHRoOiBhdXRvOyBtYXgtaGVpZ2h0
OiA2MDBweDsgb3ZlcmZsb3c6IGF1dG87IGJhY2tncm91bmQtY29sb3I6IHZhcigtLWRhcmty
ZWFkZXItYmctLWhpZ2hsaWdodC1iZyk7IGJvcmRlci1yYWRpdXM6IDVweDsgY29sb3I6IHZh
cigtLWRhcmtyZWFkZXItdGV4dC0taGlnaGxpZ2h0LWNvbG9yKTsgb3ZlcmZsb3ctd3JhcDog
bm9ybWFsOyBsZXR0ZXItc3BhY2luZzogbm9ybWFsOyBvcnBoYW5zOiAyOyB0ZXh0LWFsaWdu
OiBsZWZ0OyB0ZXh0LWluZGVudDogMHB4OyB0ZXh0LXRyYW5zZm9ybTogbm9uZTsgd2lkb3dz
OiAyOyB3b3JkLXNwYWNpbmc6IDBweDsgLXdlYmtpdC10ZXh0LXN0cm9rZS13aWR0aDogMHB4
OyB0ZXh0LWRlY29yYXRpb24tdGhpY2tuZXNzOiBpbml0aWFsOyB0ZXh0LWRlY29yYXRpb24t
c3R5bGU6IGluaXRpYWw7IHRleHQtZGVjb3JhdGlvbi1jb2xvcjogaW5pdGlhbDsiPjxjb2Rl
IHN0eWxlPSJtYXJnaW46IDBweDsgcGFkZGluZzogMHB4OyBib3JkZXI6IDBweDsgZm9udC1z
dHlsZTogaW5oZXJpdDsgZm9udC12YXJpYW50OiBpbmhlcml0OyBmb250LXdlaWdodDogaW5o
ZXJpdDsgZm9udC1zdHJldGNoOiBpbmhlcml0OyBsaW5lLWhlaWdodDogaW5oZXJpdDsgZm9u
dC1mYW1pbHk6IGluaGVyaXQ7IGZvbnQtc2l6ZTogMTNweDsgdmVydGljYWwtYWxpZ246IGJh
c2VsaW5lOyBib3gtc2l6aW5nOiBpbmhlcml0OyBiYWNrZ3JvdW5kLWNvbG9yOiB0cmFuc3Bh
cmVudDsgd2hpdGUtc3BhY2U6IGluaGVyaXQ7Ij4gICAgPHNwYW4gY2xhc3M9ImhsanMta2V5
d29yZCIgc3R5bGU9Im1hcmdpbjogMHB4OyBwYWRkaW5nOiAwcHg7IGJvcmRlcjogMHB4OyBm
b250LXN0eWxlOiBpbmhlcml0OyBmb250LXZhcmlhbnQ6IGluaGVyaXQ7IGZvbnQtd2VpZ2h0
OiBpbmhlcml0OyBmb250LXN0cmV0Y2g6IGluaGVyaXQ7IGxpbmUtaGVpZ2h0OiBpbmhlcml0
OyBmb250LWZhbWlseTogaW5oZXJpdDsgZm9udC1zaXplOiAxM3B4OyB2ZXJ0aWNhbC1hbGln
bjogYmFzZWxpbmU7IGJveC1zaXppbmc6IGluaGVyaXQ7IGNvbG9yOiB2YXIoLS1kYXJrcmVh
ZGVyLXRleHQtLWhpZ2hsaWdodC1rZXl3b3JkKTsiPnR5cGVkZWY8L3NwYW4+IGxleDo6bGV4
ZXJ0bDo6dG9rZW4mbHQ7PHNwYW4gY2xhc3M9ImhsanMta2V5d29yZCIgc3R5bGU9Im1hcmdp
bjogMHB4OyBwYWRkaW5nOiAwcHg7IGJvcmRlcjogMHB4OyBmb250LXN0eWxlOiBpbmhlcml0
OyBmb250LXZhcmlhbnQ6IGluaGVyaXQ7IGZvbnQtd2VpZ2h0OiBpbmhlcml0OyBmb250LXN0
cmV0Y2g6IGluaGVyaXQ7IGxpbmUtaGVpZ2h0OiBpbmhlcml0OyBmb250LWZhbWlseTogaW5o
ZXJpdDsgZm9udC1zaXplOiAxM3B4OyB2ZXJ0aWNhbC1hbGlnbjogYmFzZWxpbmU7IGJveC1z
aXppbmc6IGluaGVyaXQ7IGNvbG9yOiB2YXIoLS1kYXJrcmVhZGVyLXRleHQtLWhpZ2hsaWdo
dC1rZXl3b3JkKTsiPmNoYXI8L3NwYW4+IDxzcGFuIGNsYXNzPSJobGpzLWtleXdvcmQiIHN0
eWxlPSJtYXJnaW46IDBweDsgcGFkZGluZzogMHB4OyBib3JkZXI6IDBweDsgZm9udC1zdHls
ZTogaW5oZXJpdDsgZm9udC12YXJpYW50OiBpbmhlcml0OyBmb250LXdlaWdodDogaW5oZXJp
dDsgZm9udC1zdHJldGNoOiBpbmhlcml0OyBsaW5lLWhlaWdodDogaW5oZXJpdDsgZm9udC1m
YW1pbHk6IGluaGVyaXQ7IGZvbnQtc2l6ZTogMTNweDsgdmVydGljYWwtYWxpZ246IGJhc2Vs
aW5lOyBib3gtc2l6aW5nOiBpbmhlcml0OyBjb2xvcjogdmFyKC0tZGFya3JlYWRlci10ZXh0
LS1oaWdobGlnaHQta2V5d29yZCk7Ij5jb25zdDwvc3Bhbj4qLCBib29zdDo6bXBsOjp2ZWN0
b3ImbHQ7bGV4OjpvbWl0LCBzdGQ6OnN0cmluZyZndDssIGJvb3N0OjptcGw6OnRydWVfJmd0
OyB0b2tlbl90eXBlOwo8L2NvZGU+PC9wcmU+CiAgICA8cD5JJ20gcHJldHR5IHN1cmUgSSBo
YXZlIHNldmVyYWwgYW5zd2VycyBzaG93aW5nIHN1Y2ggbGV4ZXJzIG9uCiAgICAgIFN0YWNr
b3ZlcmZsb3c6IDxhIGNsYXNzPSJtb3otdHh0LWxpbmstZnJlZXRleHQiIGhyZWY9Imh0dHBz
Oi8vc3RhY2tvdmVyZmxvdy5jb20vc2VhcmNoP3E9bGV4K3Rva2VuK29taXQiPmh0dHBzOi8v
c3RhY2tvdmVyZmxvdy5jb20vc2VhcmNoP3E9bGV4K3Rva2VuK29taXQ8L2E+CiAgICAgICh0
aG9zZSBrZXl3b3JkcyBpbnZhcmlhYmx5IGFwcGVhciBpbiB0aGlzIGtpbmQgb2YgZGVjbGFy
YXRpb24pLjwvcD4KICAgIDxwPkkgaG9wZSB5b3Ugd2lsbCBmaW5kIHRoaW5ncyBoZXJlIHRo
YXQgZWl0aGVyIGFuc3dlciB5b3VyIHF1ZXN0aW9uCiAgICAgIG9yIGhlbHAgeW91IG1ha2Ug
aXQgbW9yZSBzcGVjaWZpYyw8L3A+CiAgICA8cD5SZWdhcmRzLDwvcD4KICAgIDxwPlNldGg8
L3A+CiAgICA8cD5BIHByb3BvcyBvZiBub3RoaW5nOiBJIGNvdWxkIGhhdmUgYSBsb29rIGF0
IHRoZSBhY3R1YWwgd29yay4gSQogICAgICB1c3VhbGx5IHJlY29tbWVuZCBnb2luZyB3aXRo
b3V0IExleDxicj4KICAgIDwvcD4KICAgIDxibG9ja3F1b3RlIHR5cGU9ImNpdGUiCmNpdGU9
Im1pZDpDQUs9cW1rNzNWLXJjZjBCb0hIay1HeTdXdkhOMXpnMkMwQ2dKNU1YbncweEZrakw9
UEFAbWFpbC5nbWFpbC5jb20iPgogICAgICA8ZGl2IGRpcj0ibHRyIj4KICAgICAgICA8ZGl2
Pjxicj4KICAgICAgICA8L2Rpdj4KICAgICAgICA8ZGl2PlRoYW5rcyw8L2Rpdj4KICAgICAg
ICA8ZGl2PkJyaWFuPGJyPgogICAgICAgIDwvZGl2PgogICAgICAgIDxkaXY+PGJyPgogICAg
ICAgICAgLS0gPGJyPgogICAgICAgICAgPGRpdiBkaXI9Imx0ciIgY2xhc3M9ImdtYWlsX3Np
Z25hdHVyZSIKICAgICAgICAgICAgZGF0YS1zbWFydG1haWw9ImdtYWlsX3NpZ25hdHVyZSI+
CiAgICAgICAgICAgIDxkaXYgZGlyPSJsdHIiPjxiPkJyaWFuIEUuIE1vcm96PC9iPgogICAg
ICAgICAgICAgIDxkaXY+CiAgICAgICAgICAgICAgICA8ZGl2PjxpPjxhIGhyZWY9Im1haWx0
bzpicmltb3JvekBnbWFpbC5jb20iCiAgICAgICAgICAgICAgICAgICAgICB0YXJnZXQ9Il9i
bGFuayIgbW96LWRvLW5vdC1zZW5kPSJ0cnVlIj5icmltb3JvekBnbWFpbC5jb208L2E+PC9p
PjwvZGl2PgogICAgICAgICAgICAgICAgPGRpdj48aT48YSBocmVmPSJtYWlsdG86Ym1vcm96
QGNzM2NvbXB1dGluZy5jb20iCiAgICAgICAgICAgICAgICAgICAgICB0YXJnZXQ9Il9ibGFu
ayIgbW96LWRvLW5vdC1zZW5kPSJ0cnVlIj5ibW9yb3pAY3MzY29tcHV0aW5nLmNvbTwvYT48
L2k+PC9kaXY+CiAgICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgIDwvZGl2PgogICAg
ICAgICAgPC9kaXY+CiAgICAgICAgPC9kaXY+CiAgICAgIDwvZGl2PgogICAgICA8YnI+CiAg
ICAgIDxmaWVsZHNldCBjbGFzcz0ibWltZUF0dGFjaG1lbnRIZWFkZXIiPjwvZmllbGRzZXQ+
CiAgICAgIDxicj4KICAgICAgPGZpZWxkc2V0IGNsYXNzPSJtaW1lQXR0YWNobWVudEhlYWRl
ciI+PC9maWVsZHNldD4KICAgICAgPHByZSBjbGFzcz0ibW96LXF1b3RlLXByZSIgd3JhcD0i
Ij5fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpTcGly
aXQtZ2VuZXJhbCBtYWlsaW5nIGxpc3QKPGEgY2xhc3M9Im1vei10eHQtbGluay1hYmJyZXZp
YXRlZCIgaHJlZj0ibWFpbHRvOlNwaXJpdC1nZW5lcmFsQGxpc3RzLnNvdXJjZWZvcmdlLm5l
dCI+U3Bpcml0LWdlbmVyYWxAbGlzdHMuc291cmNlZm9yZ2UubmV0PC9hPgo8YSBjbGFzcz0i
bW96LXR4dC1saW5rLWZyZWV0ZXh0IiBocmVmPSJodHRwczovL2xpc3RzLnNvdXJjZWZvcmdl
Lm5ldC9saXN0cy9saXN0aW5mby9zcGlyaXQtZ2VuZXJhbCI+aHR0cHM6Ly9saXN0cy5zb3Vy
Y2Vmb3JnZS5uZXQvbGlzdHMvbGlzdGluZm8vc3Bpcml0LWdlbmVyYWw8L2E+CjwvcHJlPgog
ICAgPC9ibG9ja3F1b3RlPgogICAgPHA+PGJyPgogICAgPC9wPgogIDwvYm9keT4KPC9odG1s
Pgo=
--------------2EF49C2590A42D896AE41ADD--


--===============3922784831741248047==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline


--===============3922784831741248047==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Spirit-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/spirit-general

--===============3922784831741248047==--