RE: wiktionary parser
"Shawn Wildermuth" <[email protected]> Sun, 3 Feb 2013 19:26:34 -0500
| Newsgroups | gmane.comp.windows.off-topic |
|---|---|
| Message-ID | <[email protected]> |
------=_NextPart_000_0328_01CE0244.61082B00 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable +1 on HTML AgilityPack, I=92ve used it to scrap Xbox.com a bunch of times. =20 - stw =20 I don=92t even use spell check =96 my misspellings are on purpose. =20 From: [email protected] [mailto:[email protected]] On Behalf Of Christopher Pelatari Sent: Sunday, February 3, 2013 7:25 PM To: [email protected] Subject: Re: [OT] wiktionary parser =20 =20=20 http://htmlagilitypack.codeplex.com/ On Sunday, February 3, 2013, Thomas wrote: =20=20 Three years ago I needed to do some web page scraping for a project that I was working on. I looked for an open source .Net solution to do it, but couldn't find one. I ended up using a an open source Java project called HtmlUnit. HtmlUnit implements a "headless browser". It gives you a DOM view of the HTML page, and lets you find elements using XPath syntax. But of course HtmlUnit is Java code, and you can't just call Java code from a .Net application. That's where IKVM.NET <http://IKVM.NET> comes in. IKVM.NET <http://IKVM.NET> is a an implementation of Java for the .Net Framework. It includes a Java Virtual Machine that runs under the .Net CLR, a .Net implementation of the Java libraries, and a program to convert Java JAR files to .Net assemblies. So when you're done, you deploy .Net assemblies. You don't need to install Java on your target machine.=20 This blog post explains how to use HtmlUnit in a .Net program using IKVM.NE= T <http://IKVM.NET> : http://blog.stevensanderson.com/2010/03/30/using-htmlunit-on-net-for-headle= s s-browser-automation/ One drawback of using HtmlUnit this way is that when you are writing code that uses the HtmlUnit API, you need to live in two worlds at once. You're writing C# (or VB.Net) code, but HtmlUnit uses Java naming conventions. It also uses Java listeners rather than .Net events. So you may need to write Java-style listeners in C#. Also, the documentation is in Javadoc format. I= t helps if you have some Java experience. I know that this all seems like an ugly kludge. But it works well. HtmlUnit and IKVM.NET <http://IKVM.NET> are mature, well-supported projects. Both are hosted on SourceForge. Using them together is a lot less work than writing your own HTML parser. Tom --- In [email protected] <mailto:[email protected]> , Adrian Vintu wrote: > > Thank you Frank, but I only have VS 2010, Win 7. I also need not WinRT code to post on shared server. >=20 > Best regards, > Adrian >=20 >=20 > ________________________________ > From: Frank Lavigne=20 > To: Adrian Vintu ; "[email protected] <mailto:[email protected]> "=20 > Sent: Friday, February 1, 2013 2:14 PM > Subject: RE: [OT] wiktionary parser >=20 >=20 > =C2=20=20 > Hi, >=20 > I wrote a screen scraping utility that might help. >=20 > It's open source and a free download. >=20 > Thanks, > Frank >=20 >=20 > http://code.msdn.microsoft.com/windowsapps/Screen-Scraping-Utility-11e3d457 >=20 > Sent from my Windows Phone >=20 > ________________________________ > From: Adrian Vintu > Sent: 2/1/2013 4:13 AM > To: [email protected] <mailto:[email protected]>=20 > Subject: [OT] wiktionary parser >=20 > =C2=20=20 > Hi guys, >=20 > I have been looking for some C# Wiktionary parser but just cannot get my hands on an easy to use dll. >=20 > I have found only half solutions, or very complex ways of extracting data= . >=20 > I could use a parser that either scraps the html pages, or looks into the downloadable en/de/fr wiki pages. >=20 > Any ideas? >=20 > Thank you, >=20 > Best regards, > Adrian > --=20 -- -Christopher Hendrix Pelatari | Vice President - Velocity Databank, Inc. ------=_NextPart_000_0328_01CE0244.61082B00 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/htm= l4/strict.dtd"> <html> <head> </head> <body style=3D"background-color: #fff;"> <span style=3D"display:none"> </span> <!--~-|**|PrettyHtmlStartT|**|-~--> <div id=3D"ygrp-mlmsg" style=3D"position:relative;"> <div id=3D"ygrp-msg" style=3D"z-index: 1;"> <!--~-|**|PrettyHtmlEndT|**|-~--> <div id=3D"ygrp-text" > =20=20=20=20=20=20 =20=20=20=20=20=20 <p> <div class=3D"WordSection1"><p class=3D"MsoNormal"><span style=3D"font-size= :11.0pt;color:#1F497D;">+1 on HTML AgilityPack, I’ve used it to s= crap Xbox.com a bunch of times.<o></o></span></p><p class=3D"MsoNormal"><sp= an style=3D"font-size:11.0pt;color:#1F497D;"><o> </o></span></p><div><= p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;color:#1F497D;">- stw= <o></o></span></p><p class=3D"MsoNormal"><span style=3D"font-size:11.0pt;co= lor:#1F497D;"><o> </o></span></p><p class=3D"MsoNormal"><span style=3D= "font-size:11.0pt;color:#1F497D;">I don’t even use spell check –= ; my misspellings are on purpose.<o></o></span></p></div><p class=3D"MsoNor= mal"><span style=3D"font-size:11.0pt;color:#1F497D;"><o> </o></span></= p><div><div style=3D"border-top:solid #E1E1E1 1.0pt;"><p class=3D"MsoNormal= "><b><span style=3D"font-size:11.0pt;">From:</span></b><span style=3D"font-= size:11.0pt;"> [email protected] [mailto:win_tech_off_topi= [email protected]] <b>On Behalf Of </b>Christopher Pelatari<br><b>Sent:</b>= Sunday, February 3, 2013 7:25 PM<br><b>To:</b> win_tech_off_topic@yahoogro= ups.com<br><b>Subject:</b> Re: [OT] wiktionary parser<o></o></span></p></di= v></div><p class=3D"MsoNormal"><o> </o></p><p class=3D"MsoNormal">&nbs= p; <o></o></p><div id=3D"ygrp-mlmsg"><div id=3D"ygrp-msg"><div id=3D"ygrp-t= ext"><p><a href=3D"http://htmlagilitypack.codeplex.com/">http://htmlagility= pack.codeplex.com/</a><br><br>On Sunday, February 3, 2013, Thomas wrote:<o>= </o></p><blockquote style=3D"border-left:solid #CCCCCC 1.0pt;"><div><p clas= s=3D"MsoNormal"> <o></o></p><div><div><div><p>Three years ago I neede= d to do some web page scraping for a project that I was working on. I looke= d for an open source .Net solution to do it, but couldn't find one. I ended= up using a an open source Java project called HtmlUnit. HtmlUnit implement= s a "headless browser". It gives you a DOM view of the HTML page,= and lets you find elements using XPath syntax.<br><br>But of course HtmlUn= it is Java code, and you can't just call Java code from a .Net application.= That's where <a href=3D"http://IKVM.NET" target=3D"_blank">IKVM.NET</a> co= mes in. <a href=3D"http://IKVM.NET" target=3D"_blank">IKVM.NET</a> is a an = implementation of Java for the .Net Framework. It includes a Java Virtual M= achine that runs under the .Net CLR, a .Net implementation of the Java libr= aries, and a program to convert Java JAR files to .Net assemblies. So when = you're done, you deploy .Net assemblies. You don't need to install Java on = your target machine. <br><br>This blog post explains how to use HtmlUnit in= a .Net program using <a href=3D"http://IKVM.NET" target=3D"_blank">IKVM.NE= T</a>:<br><br><a href=3D"http://blog.stevensanderson.com/2010/03/30/using-h= tmlunit-on-net-for-headless-browser-automation/" target=3D"_blank">http://b= log.stevensanderson.com/2010/03/30/using-htmlunit-on-net-for-headless-brows= er-automation/</a><br><br>One drawback of using HtmlUnit this way is that w= hen you are writing code that uses the HtmlUnit API, you need to live in tw= o worlds at once. You're writing C# (or VB.Net) code, but HtmlUnit uses Jav= a naming conventions. It also uses Java listeners rather than .Net events. = So you may need to write Java-style listeners in C#. Also, the documentatio= n is in Javadoc format. It helps if you have some Java experience.<br><br>I= know that this all seems like an ugly kludge. But it works well. HtmlUnit = and <a href=3D"http://IKVM.NET" target=3D"_blank">IKVM.NET</a> are mature, = well-supported projects. Both are hosted on SourceForge. Using them togethe= r is a lot less work than writing your own HTML parser.<br><br>Tom<br><br>-= -- In <a href=3D"mailto:[email protected]">win_tech_off_to= [email protected]</a>, Adrian Vintu wrote:<br>><br>> Thank you Fran= k, but I only have VS 2010, Win 7. I also need not WinRT code to post on sh= ared server.<br>> <br>> Best regards,<br>> Adrian<br>> <br>>= <br>> ________________________________<br>> From: Frank Lavigne <br>= > To: Adrian Vintu ; "<a href=3D"mailto:win_tech_off_topic@yahoogro= ups.com">[email protected]</a>" <br>> Sent: Friday= , February 1, 2013 2:14 PM<br>> Subject: RE: [OT] wiktionary parser<br>&= gt; <br>> <br>> =C2 <br>> Hi,<br>> <br>> I wrote a scr= een scraping utility that might help.<br>> <br>> It's open source and= a free download.<br>> <br>> Thanks,<br>> Frank<br>> <br>> <= br>> <a href=3D"http://code.msdn.microsoft.com/windowsapps/Screen-Scrapi= ng-Utility-11e3d457" target=3D"_blank">http://code.msdn.microsoft.com/windo= wsapps/Screen-Scraping-Utility-11e3d457</a><br>> <br>> Sent from my W= indows Phone<br>> <br>> ________________________________<br>> From= : Adrian Vintu<br>> Sent: 2/1/2013 4:13 AM<br>> To: <a href=3D"mailto= :[email protected]">[email protected]</a>= <br>> Subject: [OT] wiktionary parser<br>> <br>> =C2 <br>>= ; Hi guys,<br>> <br>> I have been looking for some C# Wiktionary pars= er but just cannot get my hands on an easy to use dll.<br>> <br>> I h= ave found only half solutions, or very complex ways of extracting data.<br>= > <br>> I could use a parser that either scraps the html pages, or lo= oks into the downloadable en/de/fr wiki pages.<br>> <br>> Any ideas?<= br>> <br>> Thank you,<br>> <br>> Best regards,<br>> Adrian<b= r>><o></o></p></div></div></div></div></blockquote><p class=3D"MsoNormal= "><br><br>-- <br>--<br>-Christopher Hendrix Pelatari | Vice President - Vel= ocity Databank, Inc.<o></o></p></div><div><p class=3D"MsoNormal"><span styl= e=3D"color:white;"><o></o></span></p></div></div></div></div></p> </div> =20=20=20=20=20 <!--~-|**|PrettyHtmlStart|**|-~--> <div style=3D"color: #fff; height: 0;">__._,_.___</div> =20=20=20=20=20=20=20=20 =20=20 =20=20 <table cellspacing=3D4px style=3D"margin-top: 20px; margin-bottom: 10px= ;"> <tbody> <tr> <td style=3D"font-size: 12px; font-family: arial; font-weight: bo= ld; padding: 7px 5px 5px; color: #FFF; background-color: #F2F2F2; border: 1= px solid #EAEAEA " > <a style=3D"text-decoration: none; color: #2D50FD= " href=3D"http://groups.yahoo.com/group/win_tech_off_topic/post;_ylc=3DX3oD= MTJxMzRyZ2Y5BF9TAzk3MzU5NzE0BGdycElkAzM3MDE1MjQEZ3Jwc3BJZAMxNzA1MDA2NjY5BG1= zZ0lkAzYwNTAwBHNlYwNmdHIEc2xrA3JwbHkEc3RpbWUDMTM1OTkzNzU5OQ--?act=3Dreply&m= essageNum=3D60500">Reply via web post</a> </td> <td style=3D"font-size: 12px; font-family: arial; padding: 7px 5p= x 5px; color: #FFF; background-color: #F2F2F2; border: 1px solid #EAEAEA; "= > <a href=3D"mailto:[email protected]?subject=3DRE%3A%20= %5BOT%5D%20wiktionary%20parser" style=3D"text-decoration: none; color: #2D5= 0FD;"> Reply to sender </a>=20 </td> <td style=3D"font-size: 12px; font-family: arial; padding: 7px 5p= x 5px; color: #FFF; background-color: #F2F2F2; border: 1px solid #EAEAEA; "= > <a href=3D"mailto:[email protected]?subject=3D= RE%3A%20%5BOT%5D%20wiktionary%20parser" style=3D"text-decoration: none; col= or: #2D50FD"> Reply to group </a>=20 </td> <td style=3D"font-size: 12px; font-family: arial; padding: 7px 5p= x 5px; color: #FFF; background-color: #F2F2F2; border: 1px solid #EAEAEA; "= > <a href=3D"http://groups.yahoo.com/group/win_tech_off_topic/pos= t;_ylc=3DX3oDMTJldmMyM2g2BF9TAzk3MzU5NzE0BGdycElkAzM3MDE1MjQEZ3Jwc3BJZAMxNz= A1MDA2NjY5BHNlYwNmdHIEc2xrA250cGMEc3RpbWUDMTM1OTkzNzU5OQ--" style=3D"text-d= ecoration: none; color: #2D50FD">Start a New Topic</a> </td> <td style=3D"font-size: 12px; font-family: arial; padding: 7px 5p= x 5px; color: #2D50FD; background-color: #F2F2F2; border: 1px solid #EAEAEA= ; " > <a href=3D"http://groups.yahoo.com/group/win_te= ch_off_topic/message/60483;_ylc=3DX3oDMTM2ajJvcm5oBF9TAzk3MzU5NzE0BGdycElkA= zM3MDE1MjQEZ3Jwc3BJZAMxNzA1MDA2NjY5BG1zZ0lkAzYwNTAwBHNlYwNmdHIEc2xrA3Z0cGME= c3RpbWUDMTM1OTkzNzU5OQR0cGNJZAM2MDQ4Mw--" style=3D"text-decoration: none; c= olor: #2D50FD;">Messages in this topic</a> (17) </td> </tr> </tbody> </table> =20=20=20=20=20=20=20=20 <!------- Start Nav Bar ------> <!-- |**|begin egp html banner|**| --> <div id=3D"ygrp-vital" style=3D"background-color: #f2f2f2; font-family: Ver= dana; font-size: 10px; margin-bottom: 10px; padding: 10px;"> <span id=3D"vithd" style=3D"font-weight: bold; color: #333; text-tran= sform: uppercase; ">Recent Activity:</span> <ul style=3D"list-style-type: none; margin: 0; padding: 0; display: inl= ine;"> </ul> =20=20=20=20 <div style=3D"clear: both; padding-top: 2px; color: #1e66ae;"> <a href=3D"http://groups.yahoo.com/group/win_tech_off_topic;_ylc=3DX3oD= MTJla2o1NWZ1BF9TAzk3MzU5NzE0BGdycElkAzM3MDE1MjQEZ3Jwc3BJZAMxNzA1MDA2NjY5BHN= lYwN2dGwEc2xrA3ZnaHAEc3RpbWUDMTM1OTkzNzU5OQ--" style=3D"text-decoration: no= ne;">Visit Your Group</a> </div> </div> =20=20 <div id=3D"ft" style=3D"font-family: Arial; font-size: 11px; margin-top: 5p= x; padding: 0 2px 0 0; clear: both;"> <a href=3D"http://groups.yahoo.com/;_ylc=3DX3oDMTJkYWJwMXFqBF9TAzk3NDc2NT= kwBGdycElkAzM3MDE1MjQEZ3Jwc3BJZAMxNzA1MDA2NjY5BHNlYwNmdHIEc2xrA2dmcARzdGltZ= QMxMzU5OTM3NTk5" style=3D"float: left;"><img src=3D"http://l.yimg.com/a/i/u= s/yg/logo/us.gif" height=3D"15" width=3D"137" alt=3D"Yahoo! Groups" style= =3D"border: 0;"/></a> <div style=3D"color: #747575; float: right;">Switch to: <a href=3D"mailto= :win_tech_off_topic-traditional-hHKSG33TihhbjbujkaE4pw@public.gmane.org?subject=3DChange Delivery F= ormat: Traditional" style=3D"text-decoration: none;">Text-Only</a>, <a href= =3D"mailto:win_tech_off_topic-digest-hHKSG33TihhbjbujkaE4pw@public.gmane.org?subject=3DEmail Delive= ry: Digest" class=3D"margin-rt" style=3D"text-decoration: none;">Daily Dige= st</a> • <a href=3D"mailto:win_tech_off_topic-unsubscribe@yahoogroups.= com?subject=3DUnsubscribe" style=3D"text-decoration: none;">Unsubscribe</a>= • <a href=3D"http://docs.yahoo.com/info/terms/" style=3D"text-decorat= ion: none;">Terms of Use</a> • <a href=3D"mailto:ygroupsnotifications@= yahoogroups.com?subject=3DFeedback on the redesigned individual mail v1" st= yle=3D"text-decoration: none;">Send us Feedback </a></div> </div> <!-- |**|end egp html banner|**| --> </div> <!-- ygrp-msg --> <!-- Sponsor --> <!-- |**|begin egp html banner|**| --> <div id=3D"ygrp-sponsor" style=3D"width:160px; float:right; clear:none; m= argin:0 0 25px 0; background: #fff;"> <!-- Start Recommendations --> <div id=3D"ygrp-reco"> </div> <!-- End Recommendations --> </div> <!-- |**|end egp html banner|**| --> <div style=3D"clear:both; color: #FFF; font-size:1px;">.</div> </div> <img src=3D"http://geo.yahoo.com/serv?s=3D97359714/grpId=3D3701524/grpspI= d=3D1705006669/msgId=3D60500/stime=3D1359937599/nc1=3D3848642/nc2=3D5191954= /nc3=3D4836041" width=3D"1" height=3D"1"> <br> <div style=3D"color: #fff; height: 0;">__,_._,___</div> <!--~-|**|PrettyHtmlEnd|**|-~--> </body> <!--~-|**|PrettyHtmlStart|**|-~--> <head> <style type=3D"text/css"> <!-- #ygrp-mkp { border: 1px solid #d8d8d8; font-family: Arial; margin: 10px 0; padding: 0 10px; } #ygrp-mkp hr { border: 1px solid #d8d8d8; } #ygrp-mkp #hd { color: #628c2a; font-size: 85%; font-weight: 700; line-height: 122%; margin: 10px 0; } #ygrp-mkp #ads { margin-bottom: 10px; } #ygrp-mkp .ad { padding: 0 0; } #ygrp-mkp .ad p { margin: 0; } #ygrp-mkp .ad a { color: #0000ff; text-decoration: none; } #ygrp-sponsor #ygrp-lc { font-family: Arial; } #ygrp-sponsor #ygrp-lc #hd { margin: 10px 0px; font-weight: 700; font-size: 78%; line-height: 122%; } #ygrp-sponsor #ygrp-lc .ad { margin-bottom: 10px; padding: 0 0; } #actions { font-family: Verdana; font-size: 11px; padding: 10px 0; } #activity { background-color: #e0ecee; float: left; font-family: Verdana; font-size: 10px; padding: 10px; } #activity span { font-weight: 700; } #activity span:first-child { text-transform: uppercase; } #activity span a { color: #5085b6; text-decoration: none; } #activity span span { color: #ff7900; } #activity span .underline { text-decoration: underline; } .attach { clear: both; display: table; font-family: Arial; font-size: 12px; padding: 10px 0; width: 400px; } .attach div a { text-decoration: none; } .attach img { border: none; padding-right: 5px; } .attach label { display: block; margin-bottom: 5px; } .attach label a { text-decoration: none; } =20=20 blockquote { margin: 0 0 0 4px; } .bold { font-family: Arial; font-size: 13px; font-weight: 700; } .bold a { text-decoration: none; } dd.last p a { font-family: Verdana; font-weight: 700; } dd.last p span { margin-right: 10px; font-family: Verdana; font-weight: 700; } dd.last p span.yshortcuts { margin-right: 0; } div.attach-table div div a { text-decoration: none; } div.attach-table { width: 400px; } div.file-title a, div.file-title a:active, div.file-title a:hover, div.fi= le-title a:visited { text-decoration: none; } div.photo-title a, div.photo-title a:active, div.photo-title a:hover, div= .photo-title a:visited { text-decoration: none; } div#ygrp-mlmsg #ygrp-msg p a span.yshortcuts { font-family: Verdana; font-size: 10px; font-weight: normal; } .green { color: #628c2a; } .MsoNormal { margin: 0 0 0 0; } o { font-size: 0; } #photos div { float: left; width: 72px; } #photos div div { border: 1px solid #666666; height: 62px; overflow: hidden; width: 62px; } #photos div label { color: #666666; font-size: 10px; overflow: hidden; text-align: center; white-space: nowrap; width: 64px; } #reco-category { font-size: 77%; } #reco-desc { font-size: 77%; } .replbq { margin: 4px; } #ygrp-actbar div a:first-child { /* border-right: 0px solid #000;*/ margin-right: 2px; padding-right: 5px; } #ygrp-mlmsg { font-size: 13px; font-family: Arial, helvetica,clean, sans-serif; *font-size: small; *font: x-small; } #ygrp-mlmsg table { font-size: inherit; font: 100%; } #ygrp-mlmsg select, input, textarea { font: 99% Arial, Helvetica, clean, sans-serif; } #ygrp-mlmsg pre, code { font:115% monospace; *font-size:100%; } #ygrp-mlmsg * { line-height: 1.22em; } #ygrp-mlmsg #logo { padding-bottom: 10px; } #ygrp-msg p a { font-family: Verdana; } #ygrp-msg p#attach-count span { color: #1E66AE; font-weight: 700; } #ygrp-reco #reco-head { color: #ff7900; font-weight: 700; } #ygrp-reco { margin-bottom: 20px; padding: 0px; } #ygrp-sponsor #ov li a { font-size: 130%; text-decoration: none; } #ygrp-sponsor #ov li { font-size: 77%; list-style-type: square; padding: 6px 0; }=20 #ygrp-sponsor #ov ul { margin: 0; padding: 0 0 0 8px; } #ygrp-text { font-family: Georgia; } #ygrp-text p { margin: 0 0 1em 0; } #ygrp-text tt { font-size: 120%; } #ygrp-vital ul li:last-child { border-right: none !important;=20 }=20 --> </style> </head> <!--~-|**|PrettyHtmlEnd|**|-~--> </html> <!-- end group email --> ------=_NextPart_000_0328_01CE0244.61082B00--