Re: Adding 3D/Computer Vision

Well,
I'm actually attempting to model a human pilot for small aircraft (I've also posted about slowing down time, for the same project). Basically I'd like a system that can mimic human vision at a high level. Meaning, no matter what sensors I use (accessing the property tree, CV, accessing the scene graph, using gps), I have to be able to come up with ways of getting visual-like information into the "pilot". This is keeping in mind that the pilot model is psychologically realistic - it has a limited memory, that can forget, etc. Ideally, at some point I'd like to "see" planes on the runway, see the runway - be able to look out a side window if I need to refresh my memory of where I am as I'm doing a circuit.  Right now I can "see" the altitude, heading, etc, as available from the property tree. I'm currently working on head movements to look in different direction, as well as giving the pilot a sense of orientation - I use whatever tricks I can use to do so. 
I had thought of using the gps to just "know" where the runway is and I think I can get orientation information from the property tree. Doing something like that would be "cheating" if, in fact, a real pilot wouldn't be able to see the runway (unless a real pilot just "knew" where it was). 
As far as I know, all of this might be a lot easier if the simulated pilot was using instruments, as I could just read all the information from the property tree. Unfortunately, the human data I'd like to use uses a visual approach... which will probably be a lot more fun anyway :).
So, I can certainly use many tricks, as long as the information that results isn't more information than a real pilot would get. CV or exploring the scene graph are two ways I can think of to make sure that's the case :). 
Perhaps as I develop the psychology side a bit more, I might have a better idea of what minimal information I'd need to fly a circuit. At the moment, I'm just gathering my ideas (and yours apparently too :) ).  
I think one important point to note is that I'm not overly worried about processing time either, as I expect to slow down flightgear to run in 50 milisecond bursts (which I think works). 
Thanks Curtis,
Sterling

 

From: [email protected]
Date: Tue, 14 Aug 2012 14:24:49 -0500
To: [email protected]
Subject: Re: [Flightgear-users] Adding 3D/Computer Vision

On Tue, Aug 14, 2012 at 2:09 PM, Sterling Somers <[email protected]> wrote:






Curtis,
Thanks for the response. 

I have been thinking about this approach since I started the project. I used OpenCV for the original model, comparing screen captures to one another. I  was, however, rather sceptical about how much I could use OpenCV. I'm not much of a pilot but even I have trouble seeing the runway from the cockpit view. Subtracting the non-runway-colors approach might not work... That said, it might be the easiest approach, avoiding having to access any of scenery, scene graph, etc. I think if I give the AI pilot a sense of where it is, how far it's travelled, etc, I might be able to use that information to see the runway again, or at least eliminate run-way like candidates. I also came across Marcos Nieto's work on real-time lane detection. I wonder if I can use something similar to tell the pilot when the plane is more/less centred. 


I have done a bit of computer vision work over the years, but I don't think I'm what people would call an 'expert'.  That said, I can imagine quite a few challenges.  First you have to have the runway in your camera's field of view before you can do anything.  Next it could be seen from any direction or altitude or aircraft orientation and would thus need to be detected. 


You may want to think very carefully about exactly what problem you are trying to solve and what sensors you are allowed to use.
If your work is camera only, you may need to put the camera on a pan/tilt so you can find and track the runway no matter what location and orientation of the aircraft.


If you are allowed to use gps and are operating off of known runways, then you could do quite a bit just from the gps and would only need the vision system to correct for gps position (and imu attitude) errors -- on short final and through the flare.


Many people doing this sort of work use differential GPS to locate themselves within a cm or two and really don't need a vision system to correct for errors.
But GPS can be jammed or lost, so there are times a pure vision only system could be useful -- or vision combined with other non-gps sensors.


If you are looking for suitable landing places that aren't pre-defined runways, then that's more exciting I think.  Even if it's a clear runway, identifying it in a variety of situations (time of day, visibility, precipitation, etc.) from a variety of locations or camera orientations or from a distance could be quite a challenge.


What about putting the camera at the touch down point and have it find the aircraft against the sky and relay up corrections?
There are lots of things that could be done depending on exactly what problem you are trying to solve and what sensors you are allowed to use.


If it was me, I'd beg to have a gps on board. :-)
Curt.-- 
Curtis Olson:http://www.atiak.com - http://aem.umn.edu/~uav/

http://www.flightgear.org - http://gallinazo.flightgear.org


------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Flightgear-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/flightgear-users

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

_______________________________________________
Flightgear-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/flightgear-users