Re: wiktionary parser
"Thomas" <[email protected]> Sun, 03 Feb 2013 15:34:20 -0000
| Newsgroups | gmane.comp.windows.off-topic |
|---|---|
| Message-ID | <[email protected]> |
--6-Q5QDAGeKNWPseyvM3R5nrMghRn7zWZWgg19ry Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Three years ago I needed to do some web page scraping for a project that I = was working on. I looked for an open source .Net solution to do it, but cou= ldn't find one. I ended up using a an open source Java project called HtmlU= nit. HtmlUnit implements a "headless browser". It gives you a DOM view of t= he HTML page, and lets you find elements using XPath syntax. But of course HtmlUnit is Java code, and you can't just call Java code from= a .Net application. That's where IKVM.NET comes in. IKVM.NET is a an imple= mentation of Java for the .Net Framework. It includes a Java Virtual Machin= e that runs under the .Net CLR, a .Net implementation of the Java libraries= , and a program to convert Java JAR files to .Net assemblies. So when you'r= e done, you deploy .Net assemblies. You don't need to install Java on your = target machine.=20 This blog post explains how to use HtmlUnit in a .Net program using IKVM.NE= T: http://blog.stevensanderson.com/2010/03/30/using-htmlunit-on-net-for-headle= ss-browser-automation/ One drawback of using HtmlUnit this way is that when you are writing code t= hat uses the HtmlUnit API, you need to live in two worlds at once. You're w= riting C# (or VB.Net) code, but HtmlUnit uses Java naming conventions. It a= lso uses Java listeners rather than .Net events. So you may need to write J= ava-style listeners in C#. Also, the documentation is in Javadoc format. It= helps if you have some Java experience. I know that this all seems like an ugly kludge. But it works well. HtmlUnit= and IKVM.NET are mature, well-supported projects. Both are hosted on Sourc= eForge. Using them together is a lot less work than writing your own HTML p= arser. Tom --- In [email protected], Adrian Vintu wrote: > > Thank you Frank, but I only have VS 2010, Win 7. I also need not WinRT co= de to post on shared server. >=20 > Best regards, > Adrian >=20 >=20 > ________________________________ > From: Frank Lavigne=20 > To: Adrian Vintu ; "[email protected]"=20=20 > Sent: Friday, February 1, 2013 2:14 PM > Subject: RE: [OT] wiktionary parser >=20=20 >=20 > =C2=A0=20 > Hi, >=20 > I wrote a screen scraping utility that might help. >=20 > It's open source and a free download. >=20 > Thanks, > Frank >=20 >=20 > http://code.msdn.microsoft.com/windowsapps/Screen-Scraping-Utility-11e3d4= 57 >=20 > Sent from my Windows Phone >=20 > ________________________________ > From: Adrian Vintu > Sent: 2/1/2013 4:13 AM > To: [email protected] > Subject: [OT] wiktionary parser >=20 > =C2=A0=20 > Hi guys, >=20 > I have been looking for some C# Wiktionary parser but just cannot get my = hands on an easy to use dll. >=20 > I have found only half solutions, or very complex ways of extracting data= . >=20 > I could use a parser that either scraps the html pages, or looks into the= downloadable en/de/fr wiki pages. >=20 > Any ideas? >=20 > Thank you, >=20 > Best regards, > Adrian > --6-Q5QDAGeKNWPseyvM3R5nrMghRn7zWZWgg19ry Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/htm= l4/strict.dtd"> <html> <head> </head> <body style=3D"background-color: #fff;"> <span style=3D"display:none"> </span> <!--~-|**|PrettyHtmlStartT|**|-~--> <div id=3D"ygrp-mlmsg" style=3D"position:relative;"> <div id=3D"ygrp-msg" style=3D"z-index: 1;"> <!--~-|**|PrettyHtmlEndT|**|-~--> <div id=3D"ygrp-text" > =20=20=20=20=20=20 =20=20=20=20=20=20 <p>Three years ago I needed to do some web page scraping for a projec= t that I was working on. I looked for an open source .Net solution to do it= , but couldn't find one. I ended up using a an open source Java project= called HtmlUnit. HtmlUnit implements a "headless browser". It gi= ves you a DOM view of the HTML page, and lets you find elements using XPath= syntax.<br> <br> But of course HtmlUnit is Java code, and you can't just call Java code = from a .Net application. That's where IKVM.NET comes in. IKVM.NET is a = an implementation of Java for the .Net Framework. It includes a Java Virtua= l Machine that runs under the .Net CLR, a .Net implementation of the Java l= ibraries, and a program to convert Java JAR files to .Net assemblies. So wh= en you're done, you deploy .Net assemblies. You don't need to insta= ll Java on your target machine. <br> <br> This blog post explains how to use HtmlUnit in a .Net program using IKVM.NE= T:<br> <br> <a href=3D"http://blog.stevensanderson.com/2010/03/30/using-htmlunit-on-net= -for-headless-browser-automation/">http://blog.stevensanderson.com/2010/03/= 30/using-htmlunit-on-net-for-headless-browser-automation/</a><br> <br> One drawback of using HtmlUnit this way is that when you are writing code t= hat uses the HtmlUnit API, you need to live in two worlds at once. You'= re writing C# (or VB.Net) code, but HtmlUnit uses Java naming conventions. = It also uses Java listeners rather than .Net events. So you may need to wri= te Java-style listeners in C#. Also, the documentation is in Javadoc format= . It helps if you have some Java experience.<br> <br> I know that this all seems like an ugly kludge. But it works well. HtmlUnit= and IKVM.NET are mature, well-supported projects. Both are hosted on Sourc= eForge. Using them together is a lot less work than writing your own HTML p= arser.<br> <br> Tom<br> <br> --- In <a href=3D"mailto:win_tech_off_topic%40yahoogroups.com">win_tech_off= [email protected]</a>, Adrian Vintu wrote:<br> ><br> > Thank you Frank, but I only have VS 2010, Win 7. I also need not WinRT= code to post on shared server.<br> > <br> > Best regards,<br> > Adrian<br> > <br> > <br> > ________________________________<br> > From: Frank Lavigne <br> > To: Adrian Vintu ; "<a href=3D"mailto:win_tech_off_topic%40yahoog= roups.com">[email protected]</a>" <br> > Sent: Friday, February 1, 2013 2:14 PM<br> > Subject: RE: [OT] wiktionary parser<br> > <br> > <br> > =C2=A0 <br> > Hi,<br> > <br> > I wrote a screen scraping utility that might help.<br> > <br> > It's open source and a free download.<br> > <br> > Thanks,<br> > Frank<br> > <br> > <br> > <a href=3D"http://code.msdn.microsoft.com/windowsapps/Screen-Scraping-= Utility-11e3d457">http://code.msdn.microsoft.com/windowsapps/Screen-Scrapin= g-Utility-11e3d457</a><br> > <br> > Sent from my Windows Phone<br> > <br> > ________________________________<br> > From: Adrian Vintu<br> > Sent: 2/1/2013 4:13 AM<br> > To: <a href=3D"mailto:win_tech_off_topic%40yahoogroups.com">win_tech_o= [email protected]</a><br> > Subject: [OT] wiktionary parser<br> > <br> > =C2=A0 <br> > Hi guys,<br> > <br> > I have been looking for some C# Wiktionary parser but just cannot get = my hands on an easy to use dll.<br> > <br> > I have found only half solutions, or very complex ways of extracting d= ata.<br> > <br> > I could use a parser that either scraps the html pages, or looks into = the downloadable en/de/fr wiki pages.<br> > <br> > Any ideas?<br> > <br> > Thank you,<br> > <br> > Best regards,<br> > Adrian<br> ><br> <br> </p> </div> =20=20=20=20=20 <!--~-|**|PrettyHtmlStart|**|-~--> <div style=3D"color: #fff; height: 0;">__._,_.___</div> =20=20=20=20=20=20=20=20 =20=20 =20=20 <table cellspacing=3D4px style=3D"margin-top: 20px; margin-bottom: 10px= ;"> <tbody> <tr> <td style=3D"font-size: 12px; font-family: arial; font-weight: bo= ld; padding: 7px 5px 5px; color: #FFF; background-color: #F2F2F2; border: 1= px solid #EAEAEA " > <a style=3D"text-decoration: none; color: #2D50FD= " href=3D"http://groups.yahoo.com/group/win_tech_off_topic/post;_ylc=3DX3oD= MTJxMmFpbWlnBF9TAzk3MzU5NzE0BGdycElkAzM3MDE1MjQEZ3Jwc3BJZAMxNzA1MDA2NjY5BG1= zZ0lkAzYwNDk2BHNlYwNmdHIEc2xrA3JwbHkEc3RpbWUDMTM1OTkwNTY4MA--?act=3Dreply&m= essageNum=3D60496">Reply via web post</a> </td> <td style=3D"font-size: 12px; font-family: arial; padding: 7px 5p= x 5px; color: #FFF; background-color: #F2F2F2; border: 1px solid #EAEAEA; "= > <a href=3D"mailto:[email protected]?subject=3DRe%3A%20wikti= onary%20parser" style=3D"text-decoration: none; color: #2D50FD;"> Reply to sender </a>=20 </td> <td style=3D"font-size: 12px; font-family: arial; padding: 7px 5p= x 5px; color: #FFF; background-color: #F2F2F2; border: 1px solid #EAEAEA; "= > <a href=3D"mailto:[email protected]?subject=3D= Re%3A%20wiktionary%20parser" style=3D"text-decoration: none; color: #2D50FD= "> Reply to group </a>=20 </td> <td style=3D"font-size: 12px; font-family: arial; padding: 7px 5p= x 5px; color: #FFF; background-color: #F2F2F2; border: 1px solid #EAEAEA; "= > <a href=3D"http://groups.yahoo.com/group/win_tech_off_topic/pos= t;_ylc=3DX3oDMTJlc21xbjdyBF9TAzk3MzU5NzE0BGdycElkAzM3MDE1MjQEZ3Jwc3BJZAMxNz= A1MDA2NjY5BHNlYwNmdHIEc2xrA250cGMEc3RpbWUDMTM1OTkwNTY4MA--" style=3D"text-d= ecoration: none; color: #2D50FD">Start a New Topic</a> </td> <td style=3D"font-size: 12px; font-family: arial; padding: 7px 5p= x 5px; color: #2D50FD; background-color: #F2F2F2; border: 1px solid #EAEAEA= ; " > <a href=3D"http://groups.yahoo.com/group/win_te= ch_off_topic/message/60483;_ylc=3DX3oDMTM2MWE5dDFsBF9TAzk3MzU5NzE0BGdycElkA= zM3MDE1MjQEZ3Jwc3BJZAMxNzA1MDA2NjY5BG1zZ0lkAzYwNDk2BHNlYwNmdHIEc2xrA3Z0cGME= c3RpbWUDMTM1OTkwNTY4MAR0cGNJZAM2MDQ4Mw--" style=3D"text-decoration: none; c= olor: #2D50FD;">Messages in this topic</a> (13) </td> </tr> </tbody> </table> =20=20=20=20=20=20=20=20 <!------- Start Nav Bar ------> <!-- |**|begin egp html banner|**| --> <div id=3D"ygrp-vital" style=3D"background-color: #f2f2f2; font-family: Ver= dana; font-size: 10px; margin-bottom: 10px; padding: 10px;"> <span id=3D"vithd" style=3D"font-weight: bold; color: #333; text-tran= sform: uppercase; ">Recent Activity:</span> <ul style=3D"list-style-type: none; margin: 0; padding: 0; display: inl= ine;"> </ul> =20=20=20=20 <div style=3D"clear: both; padding-top: 2px; color: #1e66ae;"> <a href=3D"http://groups.yahoo.com/group/win_tech_off_topic;_ylc=3DX3oD= MTJlcHQ1ZXNrBF9TAzk3MzU5NzE0BGdycElkAzM3MDE1MjQEZ3Jwc3BJZAMxNzA1MDA2NjY5BHN= lYwN2dGwEc2xrA3ZnaHAEc3RpbWUDMTM1OTkwNTY4MA--" style=3D"text-decoration: no= ne;">Visit Your Group</a> </div> </div> =20=20 <div id=3D"ft" style=3D"font-family: Arial; font-size: 11px; margin-top: 5p= x; padding: 0 2px 0 0; clear: both;"> <a href=3D"http://groups.yahoo.com/;_ylc=3DX3oDMTJkOW40azF1BF9TAzk3MzU5Nz= E0BGdycElkAzM3MDE1MjQEZ3Jwc3BJZAMxNzA1MDA2NjY5BHNlYwNmdHIEc2xrA2dmcARzdGltZ= QMxMzU5OTA1Njgw" style=3D"float: left;"><img src=3D"http://l.yimg.com/a/i/u= s/yg/logo/us.gif" height=3D"15" width=3D"137" alt=3D"Yahoo! Groups" style= =3D"border: 0;"/></a> <div style=3D"color: #747575; float: right;">Switch to: <a href=3D"mailto= :win_tech_off_topic-traditional-hHKSG33TihhbjbujkaE4pw@public.gmane.org?subject=3DChange Delivery F= ormat: Traditional" style=3D"text-decoration: none;">Text-Only</a>, <a href= =3D"mailto:win_tech_off_topic-digest-hHKSG33TihhbjbujkaE4pw@public.gmane.org?subject=3DEmail Delive= ry: Digest" class=3D"margin-rt" style=3D"text-decoration: none;">Daily Dige= st</a> • <a href=3D"mailto:win_tech_off_topic-unsubscribe@yahoogroups.= com?subject=3DUnsubscribe" style=3D"text-decoration: none;">Unsubscribe</a>= • <a href=3D"http://docs.yahoo.com/info/terms/" style=3D"text-decorat= ion: none;">Terms of Use</a> • <a href=3D"mailto:ygroupsnotifications@= yahoogroups.com?subject=3DFeedback on the redesigned individual mail v1" st= yle=3D"text-decoration: none;">Send us Feedback </a></div> </div> <!-- |**|end egp html banner|**| --> </div> <!-- ygrp-msg --> <!-- Sponsor --> <!-- |**|begin egp html banner|**| --> <div id=3D"ygrp-sponsor" style=3D"width:160px; float:right; clear:none; m= argin:0 0 25px 0; background: #fff;"> <!-- Start Recommendations --> <div id=3D"ygrp-reco"> </div> <!-- End Recommendations --> </div> <!-- |**|end egp html banner|**| --> <div style=3D"clear:both; color: #FFF; font-size:1px;">.</div> </div> <img src=3D"http://geo.yahoo.com/serv?s=3D97359714/grpId=3D3701524/grpspI= d=3D1705006669/msgId=3D60496/stime=3D1359905680/nc1=3D5898810/nc2=3D3848643= /nc3=3D5522131" width=3D"1" height=3D"1"> <br> <div style=3D"color: #fff; height: 0;">__,_._,___</div> <!--~-|**|PrettyHtmlEnd|**|-~--> </body> <!--~-|**|PrettyHtmlStart|**|-~--> <head> <style type=3D"text/css"> <!-- #ygrp-mkp { border: 1px solid #d8d8d8; font-family: Arial; margin: 10px 0; padding: 0 10px; } #ygrp-mkp hr { border: 1px solid #d8d8d8; } #ygrp-mkp #hd { color: #628c2a; font-size: 85%; font-weight: 700; line-height: 122%; margin: 10px 0; } #ygrp-mkp #ads { margin-bottom: 10px; } #ygrp-mkp .ad { padding: 0 0; } #ygrp-mkp .ad p { margin: 0; } #ygrp-mkp .ad a { color: #0000ff; text-decoration: none; } #ygrp-sponsor #ygrp-lc { font-family: Arial; } #ygrp-sponsor #ygrp-lc #hd { margin: 10px 0px; font-weight: 700; font-size: 78%; line-height: 122%; } #ygrp-sponsor #ygrp-lc .ad { margin-bottom: 10px; padding: 0 0; } #actions { font-family: Verdana; font-size: 11px; padding: 10px 0; } #activity { background-color: #e0ecee; float: left; font-family: Verdana; font-size: 10px; padding: 10px; } #activity span { font-weight: 700; } #activity span:first-child { text-transform: uppercase; } #activity span a { color: #5085b6; text-decoration: none; } #activity span span { color: #ff7900; } #activity span .underline { text-decoration: underline; } .attach { clear: both; display: table; font-family: Arial; font-size: 12px; padding: 10px 0; width: 400px; } .attach div a { text-decoration: none; } .attach img { border: none; padding-right: 5px; } .attach label { display: block; margin-bottom: 5px; } .attach label a { text-decoration: none; } =20=20 blockquote { margin: 0 0 0 4px; } .bold { font-family: Arial; font-size: 13px; font-weight: 700; } .bold a { text-decoration: none; } dd.last p a { font-family: Verdana; font-weight: 700; } dd.last p span { margin-right: 10px; font-family: Verdana; font-weight: 700; } dd.last p span.yshortcuts { margin-right: 0; } div.attach-table div div a { text-decoration: none; } div.attach-table { width: 400px; } div.file-title a, div.file-title a:active, div.file-title a:hover, div.fi= le-title a:visited { text-decoration: none; } div.photo-title a, div.photo-title a:active, div.photo-title a:hover, div= .photo-title a:visited { text-decoration: none; } div#ygrp-mlmsg #ygrp-msg p a span.yshortcuts { font-family: Verdana; font-size: 10px; font-weight: normal; } .green { color: #628c2a; } .MsoNormal { margin: 0 0 0 0; } o { font-size: 0; } #photos div { float: left; width: 72px; } #photos div div { border: 1px solid #666666; height: 62px; overflow: hidden; width: 62px; } #photos div label { color: #666666; font-size: 10px; overflow: hidden; text-align: center; white-space: nowrap; width: 64px; } #reco-category { font-size: 77%; } #reco-desc { font-size: 77%; } .replbq { margin: 4px; } #ygrp-actbar div a:first-child { /* border-right: 0px solid #000;*/ margin-right: 2px; padding-right: 5px; } #ygrp-mlmsg { font-size: 13px; font-family: Arial, helvetica,clean, sans-serif; *font-size: small; *font: x-small; } #ygrp-mlmsg table { font-size: inherit; font: 100%; } #ygrp-mlmsg select, input, textarea { font: 99% Arial, Helvetica, clean, sans-serif; } #ygrp-mlmsg pre, code { font:115% monospace; *font-size:100%; } #ygrp-mlmsg * { line-height: 1.22em; } #ygrp-mlmsg #logo { padding-bottom: 10px; } #ygrp-msg p a { font-family: Verdana; } #ygrp-msg p#attach-count span { color: #1E66AE; font-weight: 700; } #ygrp-reco #reco-head { color: #ff7900; font-weight: 700; } #ygrp-reco { margin-bottom: 20px; padding: 0px; } #ygrp-sponsor #ov li a { font-size: 130%; text-decoration: none; } #ygrp-sponsor #ov li { font-size: 77%; list-style-type: square; padding: 6px 0; }=20 #ygrp-sponsor #ov ul { margin: 0; padding: 0 0 0 8px; } #ygrp-text { font-family: Georgia; } #ygrp-text p { margin: 0 0 1em 0; } #ygrp-text tt { font-size: 120%; } #ygrp-vital ul li:last-child { border-right: none !important;=20 }=20 --> </style> </head> <!--~-|**|PrettyHtmlEnd|**|-~--> </html> <!-- end group email --> --6-Q5QDAGeKNWPseyvM3R5nrMghRn7zWZWgg19ry--