Improvements to the SunPy database


I found the “Improvements to the SunPy database” project interesting among the ideas present in SunPy’s wiki page. This project appealed to me the most because it would integrate Fido into SunPy’s database, which in turn would really open up the SunPy’s database module to a world of possibilities.

This would enable researchers, who currently use separate queries for VSO, HEK etc. to use a common Fido client for all queries. So now, just put in whatever query you want, Fido will decide which client will best serve the query and for that client, all database functionalities, like add, download etc. will automatically be taken care of.

The second part of the project will improve the caching mechanism currently in use by the database. Currently, the caching mechanism cannot efficiently handle overlapping queries. This is because if two queries are slightly different, i.e. they have a lot of files in common, the cache will download all files again. The suggested improvement is to use a query result based caching mechanism rather than a query based mechanism that will check if the files exist for each query result. This will be very efficient in handling overlapping queries.

I have been in contact with derdon, who made the database module, for about a month now, and I now have a good understanding of how to implement both parts of this project. Looking forward to working on this!!!


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s