Webboard: Problem with national chars in URL
| Newsgroups | gmane.comp.web.mnogosearch.general |
|---|---|
| Message-ID | <[email protected]> |
Author: Alexander Barkov Email: [email protected] Message: Hi Zenon, > Perhaps I'm focusing too much on the broken URL and miss the point of your question. You wrote > > [5104]{01} Link 'http://mo▒e kiedy▒...' http://mo▒e kiedy▒.../ > [5104]{01} no Server, skip it > {sql.c:1202} Query: SELECT rec_id FROM url WHERE url='http://mo▒e kiedy▒...' > [5104]{01} PQexec: ERROR: invalid byte sequence for encoding "UTF8": 0xbf > [5104]{01} Error: 'DB err: PQexec: ERROR: invalid byte sequence for encoding "UTF8": 0xbf > > and that "indexer quits with pqexec error". > > I see two things that could potentially be bugs. For one, if indexer decides to skip a URL, it should probably skip it completely and not query the database for it. You're right. This is a bug in indexer code. It should not even try querying such URL. Looking how to fix this. > For another, indexer should probably sanitize its database queries before sending them to the database. > > I'm saying "potentially" and "probably" because I don't know the code and I can't judge what the right thing to do really is, code-wise. Then again, I wonder, if indexer is willing to submit a query to the database like the SELECT query quoted above, what would it do if it run into a URL like > > <a href="http://; DROP TABLE urlinfo">bye bye</a> > > give or take a few backslashes before the semicolon? > > Reply: <http://www.mnogosearch.org/board/message.php?id=21201>