Performance problem with regex

/*
 * This file is part of "SnipSnap Wiki/Weblog".
 *
 * Copyright (c) 2002 Stephan J. Schmidt, Matthias L. Jugel
 * All Rights Reserved.
 *
 * Please visit http://snipsnap.org/ for updates and contact.
 *
 * --LICENSE NOTICE--
 * This program is free software; you can redistribute it and/or
 * modify it under the terms of the GNU General Public License
 * as published by the Free Software Foundation; either version 2
 * of the License, or (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program; if not, write to the Free Software
 * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
 * --LICENSE NOTICE--
 */
package org.radeox.test;

import org.apache.oro.text.regex.Util;

import java.io.File;
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;

public class RegexComparison {
  public static void main(String args[]) throws IOException, org.apache.oro.text.regex.MalformedPatternException {
    File wikiTxt = new File("wiki.txt");
    BufferedReader reader = new BufferedReader(new FileReader(wikiTxt.getCanonicalFile()));
    StringBuffer input = new StringBuffer();
    String tmp;
    while ((tmp = reader.readLine()) != null) {
      input.append(tmp);
    }

    String regex = "\\{([^:}]+):?(.*?)\\}(.*?)\\{(\\1)\\}";
    oroTest(regex, input.toString());
    rexTest(regex, input.toString());

  }


  private static org.apache.oro.text.regex.PatternMatcher oroMatcher = new org.apache.oro.text.regex.Perl5Matcher();
  private static org.apache.oro.text.regex.PatternCompiler oroCompiler = new org.apache.oro.text.regex.Perl5Compiler();

  private static void oroTest(String regex, String input) throws org.apache.oro.text.regex.MalformedPatternException {
    org.apache.oro.text.regex.Pattern pattern = oroCompiler.compile(regex, org.apache.oro.text.regex.Perl5Compiler.MULTILINE_MASK);
    org.apache.oro.text.regex.Substitution substitution = new org.apache.oro.text.regex.Perl5Substitution("REPLACED");
    System.out.println("Starting jakarta.oro test");
    long millis = System.currentTimeMillis();
    for(int i = 0; i < 100; i++)
      org.apache.oro.text.regex.Util.substitute(oroMatcher, pattern, substitution, input, Util.SUBSTITUTE_ALL);
    finish(millis);
  }

  private static void rexTest(String regex, String input) {
    java.util.regex.Pattern pattern = java.util.regex.Pattern.compile(regex);
    java.util.regex.Matcher matcher = pattern.matcher(input);
    System.out.println("Starting java.util.regex test");
    long millis = System.currentTimeMillis();
    for (int i = 0; i < 100; i++)
      matcher.replaceAll("REPLACED");
    finish(millis);
  }

  private static void finish(long millis) {
    System.out.println("STOP: " + (System.currentTimeMillis() - millis)/100);
  }

}

1 Scorecard For War {anchor:Scorecard For War}




__I"m posting this section of my blog to try anad figure out why SnipSnap is taking like 20 seconds to process the page.  VERY odd  I'm not posting it here to try and get more visibility - please ignore this.  But SnipSnap is taking way to long to process it.__



Slate has a {link:great article|http://slate.msn.com/id/2080341/} talking about how to 'read' the war coverage to get a feel for how things are going

And on a side note - from Leno tonight.
{quote}
A record number of Americans watched George Bush's speech last night.

Well of course they did. It's not like they had jobs to get up and go to the next morning
{quote}
Love it

1 Those Backwards Flags {anchor:Those Backwards Flags}
I'm sure many of you have noticed that the flag patch on the right shoulder on the uniforms of our soldiers is backwards.  {link:Slate has the answer|http://slate.msn.com/id/2080338/}
{quote}
Only the flag patches affixed to right shoulders of uniforms are reversed, so the blue field of stars faces forward. (Left shoulder patches aren't a problem, as the stars face forward without meddling.) The reversal was inspired by the age-old practice of carrying flags into battle. When fastened to a standard, the American flag's blue-and-white portion is always closest to the pole. A flag bearer rushing into the fray, then, would naturally lead with the stars.
{quote}
Cool.

1 Go Daschle Go {anchor:Go Daschle Go}
Nothing like speaking from the heart to get the Republicans in a tizzy.  They've been going crazy on the news channels today over a {link:recent speech Senator Tom Daschle gave on Monday|http://www.cnn.com/2003/ALLPOLITICS/03/18/sprj.irq.congress.reax/index.html}.  'How dare he criticize the president on the eve or war'  Waa Waa Waa.  Where is it written that when war starts we suddenly lose our freedom of speech?

I thought Senator Daschle hit it dead on in a speech Monday
{quote}
\"I'm saddened, saddened that this president failed so miserably at diplomacy that we're now forced to war.\"
{quote}
He's exactly right.  Had Bush not just gone through the motions of his sham diplomacy and actually worked with the rest of the world, we could have avoided war and disarmed Saddam - after all he knew 250,000 troops were right next door.  Sure Daschle voted for war in October with Conogress.  War IF diplomacy and inspections failed.  Daschle simply believes we didn't give inspectors enough time.  I agree 100%.  We all know Bush never intended to use inspections to disarm Saddam - thats why he got more shrill the more Saddam cooperated.  It was classic.  Bush has intended to invade from DAY ONE.

But what infuriates me is these nutjob conservative pundits going crazy over Senator Daschle's comments and going so far as to say 'It's time for the politicians to __shut up__' now that war was imminent?  WTF?  This is America!  You never hear Democrats telling Republicans to stop spouting their hatred and vile.  They are entitled to their opinions.  It just amazes me how fast the people who will wrap themselves in the flag want to take away your rights under that flag because they disagree with you.  Disgusting.  Absolutely disgusting.

1 And The Cheezy Award Goes To... {anchor:And The Cheezy Award Goes To...}
As soon as Bush made his ultimatum, you had to wonder which news channel would do the cheezy thing and put up a countdown clock on screen.

And the winner is?  MSNBC, down to the second, they have a timer counting down the remaining time until Saddam's time has run out at 8PM EST Wednesday.  Come on people, its not like Saddam is just gonna vaporize at 8PM - I mean that would be great, but lets get real.  Fox News & CNN's Connie Chung get the runner up for the 'Terror Alert: Orange' banners/icons splashed on screen.

Of course what kills me is that regardless of this 'deadline' it makes more and more sense for Saddam to strike first to inflict a few casualties where he can before the Strike Eagles come streaking across the sky.  Here's a sucker bet - how many people think ALL the oil wells blow up as soon as the deadline expires?  I'd love to think our special forces troops have already seized the wells, but I can't imagine that as possible - these fields are HUGE.

1 Nothing Short Of Impressive {anchor:Nothing Short Of Impressive}
Even though I don't agree with Bush's little war, I am and always have been proud of our military.  It is absolutely amazing the planning going into this war.  Watching CNN today it was wild to hear officers talking about how the bombs were each being tweaked to change their fuse times, explosive power, etc based on their intended target, etc, etc to reduce civilian casualties.  The military has really worked hard to reduce the infamous 'collateral damage'  Of course no amount of technology is going to prevent civilian deaths in Iraq.  But it is still good to know the effort being spent on reduce the carnage.  

Beyond that, it is amazing how advanced our military has become.  From high tech bombs (JDAMs have to be one of the coolest 'upgrades' of old technology I've ever seen) to advanced fighting vehicles.  You know Saddam is toast.  While I don't agree with the decision to go to war and risk our troops, its good to know that now that the decision has been made we're going to go in with one of the best trained and precise fighting forces in the world.

It pains me to say it, but I even agree with how Secretary Rumsfeld wants to reform the military.  We don't need a huge mechanized force anymore.  We still need it, but it doesn't need to be invade Russia huge.  It needs to be repel North Korea huge.  While he has run into stiff resistance, he has had some success.  And I have to say - while he can be an arrogant SOB, I'm glad he's in charge of defense because he is constantly challenging the status quo.  He's just too much of a hawk for my liking - but oh well.

Even more important is what happens after the Marines blow through outlying cities.  The Cleanup Crew comes in to keep the peace and feed to population.  An {link:excellent article in Time Magazine|http://www.time.com/time/covers/1101030324/nsecond.html} outlines a lot of what they will do and face
{quote}
If Major Eric Murray of the 3rd Infantry Division crosses the border into Iraq, along with his M-16 rifle and 9-mm Beretta pistol he will be carrying another weapon of war: a briefcase full of cash. Just hours after the 3rd's M1A1 tanks blow through towns and villages on their way to Baghdad, Murray and his Civil Affairs Direct Support Teams will be looking to quickly spend tens of thousands of dollars to start rebuilding blown-up wells, bombed bridges and downed electricity grids. The idea, says Murray, is that instead of waiting for assistance from nongovernmental organizations, "we immediately contract the locals."
{quote}
That last line is genius.  If you're going to occupy a country - better to get the populace involved in rebuilding vs locking them down in their houses.

Of course its not always that simple.  I really hope our rebuilding phase goes smoothly, but knowing the intense tribal and religious strife common across the Middle East, I don't hold my breath.  But then I can't believe the Kurds agreed to put their troops under U.S. military command (this way its unlikely they can just seize the northen oil fields to try and declare independance.)

1 Man They Really DO Hate Us {anchor:Man They Really DO Hate Us}
{link:New poll out|http://people-press.org/reports/display.php3?ReportID=175} from the Pew Research Center.  Man, folks really DO hate our guts overseas.  The only country with a 50% or higher favorable view of the US today?  Wait for it... __Poland!__  Even Britian was less than 50% at 48%.  Most others were between the teens and thirties.  How very sad that this Texas Governor has ruined our stature in the world
\\
\\
{image:survey1.gif}

1 Michael Moore Letter To Governor Bush {anchor:Michael Moore Letter To Governor Bush}
So Michael Moore published {link:an open letter|http://www.michaelmoore.com/words/message/index.php} to 'Governor Bush'  Classic.
{quote}
So today is what you call "the moment of truth," the day that "France and the rest of world have to show their cards on the table." I'm glad to hear that this day has finally arrived. Because, I gotta tell ya, having survived 440 days of your lying and conniving, I wasn't sure if I could take much more.
{quote}
It only gets better.
{quote}
There is virtually NO ONE in America (talk radio nutters and Fox News aside) who is gung-ho to go to war. Trust me on this one. Walk out of the White House and on to any street in America and try to find five people who are PASSIONATE about wanting to kill Iraqis. YOU WON'T FIND THEM! Why? 'Cause NO Iraqis have ever come here and killed any of us!
{quote}
I'll save the rest for you to read on your own, but this made me snort milk out my nose.
{quote}
As Bill Maher said last week, how bad do you have to suck to lose a popularity contest with Saddam Hussein? The whole world is against you, Mr. Bush. Count your fellow Americans among them.
{quote}
Enjoy!

1 Being Sick .. Sucks! {anchor:Being Sick .. Sucks!}
Ugh.  You  may have noticed the blog has lit up of late and sure, the current events of the day provide plenty of material.  But I'm lucky to have a day job (been without one for months once - not pretty) and 3 kids so I hardly have time to breath.  But I've had plenty of time lately.  Came down with a wicked fever Friday night and have battled it up through this morning when it finalyl broke.  Splitting headache, fatigue, swollen glands in my neck.  Miserable.  I can honestly says for most of this weekend I was too sick to even use my laptop for more than 5 minutes or so.  I'm slowly recovering today and will probably jump back into the rat race tomorrow, but be sure my computers will be tuned in for every new detail!

Course I can't end a post without one good zing.  As all these morons talk about freedom fries and using tax dollars to {link:'rescue' American remains buried in France|http://www.cnn.com/2003/ALLPOLITICS/03/13/sprj.irq.congress.france/index.html}, I guess we should ask them how fast we should dismantle the Statue of Liberty and send it back to France where it came from?

It's amazing how quickly we forget and how far we have fallen.  Look, the French have always been a pain in the butt working to maintain their place on the world stage coming across slightly arrogant (kinda of like the U.S.)  But regardless of the superficial gripes, France and the US have always been close, yet dis-similar Allies.  Immediately after 9/11, the French proclaimed 'Today we are all Americans'  We had the world at our doorstep looking to help however they could.  What an awesome opportunity.  Now look where we are.  Having squandered all that worldy good-will, our 'President' is embarking on an illegal war and are becoming more and more isolated by the day.  What a sad day.

1 When Canada Stands You Up {anchor:When Canada Stands You Up}
you know you're screwed.  The Toronto Star has an {link:awesome article|http://www.thestar.ca/NASApp/cs/ContentServer?pagename=thestar/Layout/Article_Type1&c=Article&cid=1035779422146&call_pageid=968332188854&col=968350060724} outlining why Canada chose not to support the U.S. without U.N. approval:
{quote}
Saddam's strategy was simple. When faced with pressure, he did what the Security Council told him to do. In the language of the U.N., he complied.

By contrast, Bush appeared capricious, arrogant and ever so slightly unhinged. The more Saddam complied, the more Bush complained that he wasn't. The more successes the U.N. weapons inspectors scored in their disarmament of Iraq, the more petulant Bush became.
{quote}
When Canada is opposing your war and papers there say the President is appearing 'unhinged' you know things are bad.
{quote}
Among experts, the overwhelming consensus seems to be that there is no legal authorization for an Iraq war.
{quote}
Cool - lets try Bush on war crimes, right next to Saddam!  Sure, Saddam will get a stiffer penalty and be charged with more serious crimes, but this folly of the Shurbs is still illegal.  I just wish the US press was raising this issue more.

1 Can You Imagine... {anchor:Can You Imagine...}
If 51% of Americans are dumb enough to believe that Iraq was responsible for 9/11, can you imagine the utter chaos when we have a Red Ozone Alert during an Orange Terrorism Alert?

1 Beware The Wooden Horse {anchor:Beware The Wooden Horse}
VERY {link:interesting column|http://nytimes.com/2003/03/18/opinion/18KRIS.html} by Nicholas Kristof in the {link:NY Times|http://nytimes.com} which applies the lessons of the Iliad to todays current events.
{quote}
So, by Zeus, that third lesson from Troy is the paramount need to listen to skeptical voices. Virgil suggests that the Trojans rashly brought the wooden horse inside their city despite the alarm of two early pundits — Cassandra and Laocoon, who warned against Greeks bearing gifts. If the Trojans had just thought it over for a week, by which time the Greeks inside would have died of thirst, then the Trojan War might have ended differently (and we could all be speaking Luvian, the ancient language possibly spoken by Trojans).

But the Trojans dismissed the warnings as "windy nonsense" and sealed their fate. We Americans are the Greeks of our day, and as we now go to war, we should appreciate not only the beauty of the tale, but also the warnings within it.
{quote}

1 Paul Krugman is \"The Man\" {anchor:Paul Krugman}
Paul Krugman of the {link:New York Times|http://nytimes.com} just "gets it"  His columns in the past have been very well thought out about how Dubya's current policies could wreak our economies.  His {link:latest column|http://nytimes.com/2003/03/18/opinion/18KRUG.html} was right on and frightening:
{quote}
It's a matter of public record that this war with Iraq is largely the brainchild of a group of neoconservative intellectuals, who view it as a pilot project. In August a British official close to the Bush team told Newsweek: "Everyone wants to go to Baghdad. Real men want to go to Tehran." In February 2003, according to Ha'aretz, an Israeli newspaper, Under Secretary of State John Bolton told Israeli officials that after defeating Iraq the United States would "deal with" Iran, Syria and North Korea.

Will Iraq really be the first of many? It seems all too likely — and not only because the "Bush doctrine" seems to call for a series of wars. Regimes that have been targeted, or think they may have been targeted, aren't likely to sit quietly and wait their turn: they're going to arm themselves to the teeth, and perhaps strike first. People who really know what they are talking about have the heebie-jeebies over North Korea's nuclear program, and view war on the Korean peninsula as something that could happen at any moment. And at the rate things are going, it seems we will fight that war, or the war with Iran, or both at once, all by ourselves.
{quote}
It kills me how we have North Korea screaming like a Banchee about nuking the US in the mother of all battles, etc.  Is it bluff - a year ago I would have said yes, but these guys are activating a nuclear program in full view of the world.  Saddam lost all of his nuke program in the 90's when UN inspectors blew it all up.

So now what?
{quote}
What scares me most, however, is the home front. Look at how this war happened. There is a case for getting tough with Iraq; bear in mind that an exasperated Clinton administration considered a bombing campaign in 1998. But it's not a case that the Bush administration ever made. Instead we got assertions about a nuclear program that turned out to be based on flawed or faked evidence; we got assertions about a link to Al Qaeda that people inside the intelligence services regard as nonsense. Yet those serial embarrassments went almost unreported by our domestic news media. So most Americans have no idea why the rest of the world doesn't trust the Bush administration's motives. And once the shooting starts, the already loud chorus that denounces any criticism as unpatriotic will become deafening.
{quote}
That's right - Tom DeLay already has stated that the time for debate on war has 'passed' and nobody should criticize the administration now that war is imminent.  That's a load - I refuse to rally behind an unelected president just because he has forced a war down our throats.  Support our troops - hell yeah - the sooner they get back here the better.  But blindly support the Shurb, like hell!

1 Are We Really This Clueless? {anchor:Are We Really This Clueless?}
So, CNN had a poll they talked about yesterday (can't find the link :( ) that stated 51% of Americans believe Iraq was involved in the 9/11 attacks.  You can't be serious!  Iraq has done many things - but there has never been a shred of evidence supporting Saddam's involvement in 9/11 (if there was you can bet we'd have already invaded and I for one would have supported that invasion 100%)

Saddam is a scumbag - no question.  He tortures, rapes, kills his people and deserves to be wiped off the face of the earth.  But at what cost?  There are SO many other countries with repressive regimes that you know damn well we aren't going to 'liberate'

Believe me - I will not shed a tear when Saddam bites it.  But that doesn't change the fact that we shouldn't just be crashing into a country without world backing to overthrow the government because we want to.

I love how now we're getting these solemn reports on CNN that 'Republican Guard' units have chemical weapons and plan to use them.  Well duh!  Of course they do.  But they weren't using them!  Saddam wasn't that stupid - the inspectors would have found them - bring in some troops with the inspectors and back them up if you have to.  But now, Saddam has NOTHING to lose so of course he is going to use whatever he has.  It's his last stand and he knows it.

Kind of a self fulfilling prophecy.  Bush states we have to invade because Saddam has chemical weapons and might use them.  Saddam isn't dumb enough to use them until he's boxed in a corner - which is what Bush has done.  So thanks to Bush's obsession with invading Iraq, he's going to force Saddam into using those weapons of mass destruction.  Brilliant.

1 Why Do We Even Have A Congress? {anchor:Why Do We Even Have A Congress?}
Is it me or have the so called 'balance of powers' disappeared.  Have you noticed how more and more of Bush's policies are implemented by fiat (okay 'Presidential Order') instead of through Congress?  Why have we had no debate about war?  The President got the chicken little congress - too worried what a no vote might do to their re-election hopes - to pass him carte blanche.  Bush has sent over 300,000 troops to the Middle East to invade Iraq and there has been NO debate in Congress - no point now since they whimped out.  Whatever happened to a 'declaration of war'?  Don't you love how the administration has repeatedly refused to estimate the cost of a war with Iraq?  They are happy to send our economy over the cliff with tax cuts for the rich, but don't want to fess up how much it'll cost to invade Iraq.  Now we're told that a few days AFTER the war starts, they'll ask Congress for $90 billion to pay for it.  Like they can say no?  And yet all this time Congress has been debating Bush's proposed budget - which does NOT include any mention of the cost for war with Iraq.  What a con game!  The only good news is now that Congress knows the rough cost the Bushies want, they can take it into account.  I just hope they finally wise up, ditch the ENTIRE tax cut, make smart spending cuts, and bring our budget back into reality.  Yeah right, I can dream can't I?  So to all you rich folk with tons of dividend income - hope you enjoy the windfall.  I'll be thinking about you while I pay my bills and struggle to save enough money to pay for, maybe, the first year of college for my kids.

1 Time For The UN To Kick Some... {anchor:Time For The UN To Kick Some...}
So the US is ready to lauch a pre-emptive war against a country that has not attacked us and poses no immediate threat (and no matter what you see on FOX News, Saddam hasn't got squat that'll touch the US in any large scale fashion)  As Koffi Annan stated yesterday, a preemptive war on Iraq would __put the US in direct violation of the UN Charter__  Read that again slowly.  We're about to do something that we works decades to prevent other countries from doing - attack without being attacked.

Now, Resolution 1441 is pretty vague.  But Joshua Marshall makes an excellent point in {link:a recent post|http://talkingpointsmemo.com/march0303.html#031803211am}.  
{quote}
Now, there was a degree of willful mystification that happened here. The different parties agreed not to look too closely at each others' interpretation of what they signing. But the wording which the other countries demanded and received was wording which they believed put them in charge of deciding when or if there would be war.
{quote}
Now, I myself, no matter how upset over Bush dragging us into this stupid war, was resigned to the UN being on teh sidelines because we had seemed to weasel an attack out of the existing language of UN 1441 ('serious consequences')  But Joshua Marshall points out something VERY convincing - that the US HAD, in fact, tacitly agreed to some back to the Security Council for authorization before an attack.  When a resolution is vague, how the countries talked about it becomes paramount.
{quote}
The problem for the United States is that we pretty clearly went on the record validating this other interpretation. Here's what America's UN Representative John Negroponte said at the UN on the day the resolution passed ...

"There's no 'automaticity' and this is a two-stage process, and in that regard we have met the principal concerns that have been expressed for the resolution. Whatever violation there is, or is judged to exist, will be dealt with in the council, and the council will have an opportunity to consider the matter before any other action is taken."

What he was saying there was that 1441 was not self-enforcing. Its language and what counted as an infraction was to be decided by the Security Council. This was the price we paid for getting for getting the unanimous vote.
{quote}
Which to me says we are launching an illegal war plain an simple.  I think it's time for the UN Security Council to take up the US action and call Bush on the mat for this.  As much as Bush would love to get rid of the UN, it is NOT going to happen.  For many countries it has been a lifeline out of dispair.  And now it is seen as a way to try and counter US arrogance and power.  Yes, it, like all large diplomatic organizations, has had its share of mistakes, problems, etc.  But at the core, the UN is a vital and important institution.

So what cay they do?  Pass a resolution citing the US as being in violation of the UN Charter.  Impose sanctions.  Sure, you say.  The US wouldn't even notice.  Maybe not if France aone or some other country did this.  But imagine what would hapen if China AND Russia AND France AND Germany did this.  It would HURT.  We import too much stuff from China and sell a lot of stuff to the others.  Sure, it'll hurt them to, but nobody said sanctions are easy.  And no, it probably won't make a differnece.  But to me, its time for the UN to stop being the hand puppet of the US.  Bush played with his UN toy until it didn't do what he wanted and then he threw it away.  Thats not how it works.  But if the UN sits by and just lets it happen, they WILL have lost all authority.  Only by standing up to the US can they show they are still an important and legitimate institution.

Oh and you'll love this.  Word has it that Bush, after his war is over and Iraq needs to be rebuilt, wants to call on the rest of the world and the UN to come in and rebuild things.  WTF?  Thats bold.  You basically tell the entire world to FO and then expect them to come in with billions of dollars and man power to clean up the mess you created!  You wish they could tell Bush to FO and pay for it himself.  But Bush will simply cry poverty (due to tax and spend Democrats of course) and say he can't pay for the rebuildin and start pulling out.  Then the UN will have no choice but to take over to prevent the country from falling into chaos.  Kinda like what happened in Afghanistan.  Notice how the US totally ignored rebuilding that country?  Pretty sad if you ask me, and selfish.