"datamining" thousands of IGC files

Hi,

are there any open/free tools to datamine on database of many IGC files?

There is a threat that the free airspace for gliding in Slovakia is going to be significantly restricted. We need tools which help prepare arguments about airspace usage and frequency of our flights.

If I was starting from scratch I would probably try to import all the flights into a GIS database (PostGIS) and do the data cleaning and queries there. Does anybody have experience with this?

Thanks for you help,
Tibor Arpas

I have an intern currently working on a slightly different project for US flights in an effort to isolate for potential cloud flying incedents over thousands of competition flights. It has been very interesting so far. More later. He did create a batch method for adding large sets of flights (but only a few dozen at a time). Not sure what dbase he is using.

Send me your email and I will connect you.

On Monday, March 5, 2012 12:26:07 PM UTC-5, Tibor wrote:
Hi,

are there any open/free tools to datamine on database of many IGC files?

There is a threat that the free airspace for gliding in Slovakia is going to be significantly restricted. We need tools which help prepare arguments about airspace usage and frequency of our flights.

If I was starting from scratch I would probably try to import all the flights into a GIS database (PostGIS) and do the data cleaning and queries there. Does anybody have experience with this?

Thanks for you help,
Tibor Arpas

June 2010 Soaring Page 32

There is an article on Thermal Routes by Alfred Ultsch. I didn't see any contact info, but it might get you started.


On Monday, March 5, 2012 9:26:07 AM UTC-8, Tibor wrote:
Hi,

are there any open/free tools to datamine on database of many IGC files?

There is a threat that the free airspace for gliding in Slovakia is going to be significantly restricted. We need tools which help prepare arguments about airspace usage and frequency of our flights.

If I was starting from scratch I would probably try to import all the flights into a GIS database (PostGIS) and do the data cleaning and queries there. Does anybody have experience with this?

Thanks for you help,
Tibor Arpas

Hi Tibor,
did you already contact Christian at www.streckenflug.at.
He showed us the glider routes in Austria.
Since the pilots from Vienna often fly in your country, they might be verry
willing to help to fight airspace restrictions
Regards and good success
Walter

"Tibor" schrieb im Newsbeitrag
news:28104876.3949.1330968367200.JavaMail.geo-discussion-forums@vbbed8...
Hi,

are there any open/free tools to datamine on database of many IGC files?

There is a threat that the free airspace for gliding in Slovakia is going
to be significantly restricted. We need tools which help prepare arguments
about airspace usage and frequency of our flights.

If I was starting from scratch I would probably try to import all the
flights into a GIS database (PostGIS) and do the data cleaning and queries
there. Does anybody have experience with this?

Thanks for you help,
Tibor Arpas

At 17:26 05 March 2012, Tibor wrote:
Hi,

are there any open/free tools to datamine on database of many IGC files?

There is a threat that the free airspace for gliding in Slovakia is going
to be significantly restricted. We need tools which help prepare
arguments
about airspace usage and frequency of our flights.

If I was starting from scratch I would probably try to import all the
flights into a GIS database (PostGIS) and do the data cleaning and
queries
there. Does anybody have experience with this?

Thanks for you help,
Tibor Arpas


Hi Tibor,

I have done some of this with the IGC files (about 8000 of them I recall)
from the British Gliding Association national ladder. It was published
recently in Sailplane and Gliding and I'll willingly help you in any way I
can. There are some animations to look at if you visit:

http://www.gliding.co.uk/sailplanean...ticles.htm#SG1

The same approach was used to look at all flights over FL100 to back up a
(successful) request for exemption from transponder carriage. I'll put this
on Flickr in a minute.

All this was done by brute force with PERL code, taking a day to run over
the IGC files. The visualisation was done in 3DEM which I highly recommend
for this task.

None of this is open source - though I'd be happy to open source my code if
it is any use.

Please contact me off list and I'll explain further.

Paul

Tibor wrote:
There is a threat that the free airspace for gliding in Slovakia is
going to be significantly restricted. We need tools which help
prepare arguments about airspace usage and frequency of our flights.

One could use the XCSoar calculation engine to extract data from bulk
IGC files. What exactly do you need?

The biggest problem with bulk IGC files is how to get so many IGC
files. The OLC is a commercial company and bans you for attempting to
download more than 10 files in one day. Therefore, supporting the OLC
is harmful for projects like yours.

Max

At 17:26 05 March 2012, Tibor wrote:
Hi,

are there any open/free tools to datamine on database of many IGC files?

There is a threat that the free airspace for gliding in Slovakia is going
to be significantly restricted. We need tools which help prepare
arguments
about airspace usage and frequency of our flights.

If I was starting from scratch I would probably try to import all the
flights into a GIS database (PostGIS) and do the data cleaning and
queries
there. Does anybody have experience with this?

Thanks for you help,
Tibor Arpas


[Not sure why the subject didn't appear on this post before]

Hi Tibor,

I have done some of this with the IGC files (about 8000 of them I recall)
from the British Gliding Association national ladder. It was published
recently in Sailplane and Gliding and I'll willingly help you in any way I
can. There are some animations to look at if you visit:

http://www.gliding.co.uk/sailplanean...ticles.htm#SG1

The same approach was used to look at all flights over FL100 to back up a
(successful) request for exemption from transponder carriage. I'll put this
on Flickr in a minute.

All this was done by brute force with PERL code, taking a day to run over
the IGC files. The visualisation was done in 3DEM which I highly recommend
for this task.

None of this is open source - though I'd be happy to open source my code if
it is any use.

Please contact me off list and I'll explain further.

Paul

At 17:26 05 March 2012, Tibor wrote:
Hi,

are there any open/free tools to datamine on database of many IGC files?

There is a threat that the free airspace for gliding in Slovakia is going
to be significantly restricted. We need tools which help prepare
arguments
about airspace usage and frequency of our flights.

If I was starting from scratch I would probably try to import all the
flights into a GIS database (PostGIS) and do the data cleaning and
queries
there. Does anybody have experience with this?

Thanks for you help,
Tibor Arpas


[Not sure why the subject didn't appear on this post before]

Hi Tibor,

I have done some of this with the IGC files (about 8000 of them I recall)
from the British Gliding Association national ladder. It was published
recently in Sailplane and Gliding and I'll willingly help you in any way I
can. There are some animations to look at if you visit:

http://www.gliding.co.uk/sailplanean...ticles.htm#SG1

The same approach was used to look at all flights over FL100 to back up a
(successful) request for exemption from transponder carriage. I'll put this
on Flickr in a minute.

All this was done by brute force with PERL code, taking a day to run over
the IGC files. The visualisation was done in 3DEM which I highly recommend
for this task.

None of this is open source - though I'd be happy to open source my code if
it is any use.

Please contact me off list and I'll explain further.

Paul

At 17:26 05 March 2012, Tibor wrote:
Hi,

are there any open/free tools to datamine on database of many IGC files?

There is a threat that the free airspace for gliding in Slovakia is going
to be significantly restricted. We need tools which help prepare
arguments
about airspace usage and frequency of our flights.

If I was starting from scratch I would probably try to import all the
flights into a GIS database (PostGIS) and do the data cleaning and
queries
there. Does anybody have experience with this?

Thanks for you help,
Tibor Arpas


[Not sure why the subject didn't appear on this post before]

Hi Tibor,

I have done some of this with the IGC files (about 8000 of them I recall)
from the British Gliding Association national ladder. It was published
recently in Sailplane and Gliding and I'll willingly help you in any way I
can. There are some animations to look at if you visit:

http://www.gliding.co.uk/sailplanean...ticles.htm#SG1

The same approach was used to look at all flights over FL100 to back up a
(successful) request for exemption from transponder carriage. I'll put this
on Flickr in a minute.

All this was done by brute force with PERL code, taking a day to run over
the IGC files. The visualisation was done in 3DEM which I highly recommend
for this task.

None of this is open source - though I'd be happy to open source my code if
it is any use.

Please contact me off list and I'll explain further.

Paul

Thanks everyone for suggestions, please keep them coming . For now we need to answer simple question or two. e.g. how many "flights/hours have reached 7500ft MSL."?

We'll probably get only competition flights (no online competitions), as we luckily have quite enough competitions in Slovakia and it's easy to approach the organizers.

For longer term, of course we would like to have more sophisticated collection, filtering and ad hoc queries. That's why I mentioned importing flights into relational+spatial database. I think it's the propper groundwork for anybody doing statistics or visualizations. If we create something along this lines I'll let you know.

Thanks,
Tibor

On Monday, March 5, 2012 6:26:07 PM UTC+1, Tibor wrote:
Hi,

are there any open/free tools to datamine on database of many IGC files?

There is a threat that the free airspace for gliding in Slovakia is going to be significantly restricted. We need tools which help prepare arguments about airspace usage and frequency of our flights.

If I was starting from scratch I would probably try to import all the flights into a GIS database (PostGIS) and do the data cleaning and queries there. Does anybody have experience with this?

Thanks for you help,
Tibor Arpas



On Monday, March 5, 2012 6:26:07 PM UTC+1, Tibor wrote:
Hi,

are there any open/free tools to datamine on database of many IGC files?

There is a threat that the free airspace for gliding in Slovakia is going to be significantly restricted. We need tools which help prepare arguments about airspace usage and frequency of our flights.

If I was starting from scratch I would probably try to import all the flights into a GIS database (PostGIS) and do the data cleaning and queries there. Does anybody have experience with this?

Thanks for you help,
Tibor Arpas