XML python to database

[flebber]

Can anyone help me overcome a terminology and jargon barrier I am having infully defining what tools to use to fulfil a process.

I want to create a database 6 Related tables. Update information 1 or twicea week with data from an XML file that I will download, this data would update rows in 5 tables of the database. From there display, add, edit and view data in web page and produce other info based on this.

My main roadblock is the XML process, I am finding it unclear to understandwhat tools and how to manage this process. Most examples show manually inputting data.

What I know and have learnt.

- can insert and update data into database using python(values I type in)
- can query and view data with python from tables
- can design good SQL related tables(don't know much NoSQL)
- Use lxml to open view and find info from nodes of an XML file
- Basic Django/Flask/Pylons haven't completed a sizable project yet but have completed their tutorials and have some previous web experience.

When I look for info on this process

- Info from django leads ultimately to fixtures Django Docs, initial fixtures https://docs.djangoproject.com/en/dev/howto/initial-data/
- SQLAlchemy info leads http://docs.sqlalchemy.org/en/latest/orm/examples.html#xml-persistence

- Spyne RPC toolkit http://spyne.io/#s=sql&ser=Xml&show=Schema

Reading these I am not sure this covers what I am actually trying to do, reliably and repeatedly update a database with XML data. Can anyone advise of the correct terminology I should be searching for to learn more. What python tools would best help me complete

Sayth

 

[rusi]

What I know and have learnt.
- Use lxml to open view and find info from nodes of an XML file

My main roadblock is the XML process, I am finding it unclear to understand what tools and how to manage this process.
Most examples show manually inputting data.

Whats missing in lxml for this?
Maybe the 'trick' for running a web search is to look for elementtree
and then use the concepts in lxml
[Starting http://docs.python.org/2/library/xml.etree.elementtree.html ?? ]

 

[flebber]

Yes I have done the lxml search and learnt how to open view and query the file.

But what is the next step in the process? To get it so that I can reliably push XML files to my database repeatedly.

Looking for a basic structure or example to use to guide me for first time.

Sayth

 

[rusi]

Yes I have done the lxml search and learnt how to open view and query the file.
But what is the next step in the process? To get it so that I can reliably
push XML files to my database repeatedly.

Looking for a basic structure or example to use to guide me for first time.

I dont get where's the catch. Could be in any of
1. reliably
2. push xml
3. database
4. repeatedly
5. basic structure

I'm going to assume 5. So...

Forget about all the other stuff -- django, web, spyne etc
Just 2 pieces of the jigsaw to start with
- your database engine -- mysql, sqlite, postgres... whatever
- your xml input

with python as the glue inbetween.

1. So you can even whittle down the XML schema to something related to
what you finally need but relatively trivial
2. Make a correspondingly (if necessary trivialized) sql schema
3. Make a bit of python glue to read the first (as a file) and write
out the second
4. Debug the output
5. De-trivialize your simplifications
6. Fit into the larger framework

[I wrote the above as though to a somewhat inexperienced programmer.
An experienced programmer will find it condescending.
If so I apologize]