Parsing VHDL with python, where to start.

[Svenn Are Bjerkem]

Hi,
I am in the need to write an application for PyQt to visualise the
structure of a VHDL project I am working on. Looking for a sensible
way to parse VHDL files and putting them into a data structure that
PyQt can represent as a tree (or whatever the MVC is supporting)
through search engines does not give me many hints. From what I know,
VHDL is not a very easy language to parse. There seems to be a parser
for perl available, but I do not know if it is wise to use a perl
module as a template for writing something similar in python.

My initial idea is to start simple and extend features in my
application, but I fear that I may start off with wrong ideas how to
parse and then code myself into a dead-end requiring myself to rewrite
the whole application in order to get any further. I would start
finding definitions of entities and the instantiations of these and
build a tree from a set of external vhdl files stored in a file
hierarchy. If somebody have a starting point where to get going with a
task like this, I would be happy to know.

 

[c d saunter]

Svenn Are Bjerkem ([email protected]) wrote:
: Hi,
: I am in the need to write an application for PyQt to visualise the
: structure of a VHDL project I am working on. Looking for a sensible
: way to parse VHDL files and putting them into a data structure that
: PyQt can represent as a tree (or whatever the MVC is supporting)
: through search engines does not give me many hints. From what I know,
: VHDL is not a very easy language to parse. There seems to be a parser
: for perl available, but I do not know if it is wise to use a perl
: module as a template for writing something similar in python.

Hi Sven,

How much of VHDL are you looking to parse? Are you just looking at files
intended for synthesis, or at simulation/testbench files as well?

I wrote a basic parser a few years ago for automatically stringing modules
together, but it was very limited in scope. I thought VHDL was quite
simple to parse, it's very rigid. Having written a basic one I realised
I'd coded myself into a dead-end as you say. Maybe not so simple after
all...

If I started again I'd use pyparsing:
http://pyparsing.wikispaces.com/

Looks like someone is already there in part:
http://pyparsing.wikispaces.com/message/view/home/103973

Regards,
Chris

: My initial idea is to start simple and extend features in my
: application, but I fear that I may start off with wrong ideas how to
: parse and then code myself into a dead-end requiring myself to rewrite
: the whole application in order to get any further. I would start
: finding definitions of entities and the instantiations of these and
: build a tree from a set of external vhdl files stored in a file
: hierarchy. If somebody have a starting point where to get going with a
: task like this, I would be happy to know.

: --
: kind regards,
: Svenn

 

[Svenn Are Bjerkem]

How much of VHDL are you looking to parse? Are you just looking at files
intended for synthesis, or at simulation/testbench files as well?

As a start I want to parse VHDL which is going to be synthesised, and
I am limiting myself to the entities and the structural component
placement. I will drop the processes and the concurrent assignments
even if that will mask important information. It is a design viewer
tool, not a design tool that I am writing. Xilinx ISE do give me the
opportunity to browse my synthesised netlist, but there is simply too
much information.

Later the app can be extended with more functionality, depends on my
success with the initial problems that I have.

If I started again I'd use pyparsing:http://pyparsing.wikispaces.com/

Looks like someone is already there in part:http://pyparsing.wikispaces.com/message/view/home/103973

I also got a pointer to SimpleParse and now try to translate the parts
of the VHDL BNF that I need into a definition that SimpleParse can
understand. But looking at the BNF it is clear that VHDL is no easy
language to parse, just as it is no easy language to do structural
design in.

 

[Svenn Are Bjerkem]

Hi again,

when I get far enough to parse the VHDL (which is not currently the
fact, but I have to look at the work coming up downstream) I will have
to put it into an internal data structure and then write some classes
to handle the MVC between whatever data I have and the PyQt4 widget
that is going to show the block diagram. I own the book "Rapig GUI
Programming with Python and Qt" by Mark Summerfield and try to read up
on the PyQt way of doing things as I try to make a plan for my
application. I have been looking for possible data structures with
google just to find out that most of the ideas are stored in
proceedings or to conferences or ieee papers not generally available
to me. Is anybody experienced with designing data structures willing
to share some ideas with me? Since I am using the open-source version
of PyQt4, any information will eventually be available to public (if
the app pass the planning stage) if that makes helping out any
easier: There is already an app called Qucs that is running in
Qt-3 and being ported to Qt-4 that is written in C++, but I don't know
how wise it is to just reverse-engineering C++ classes and translate
them into python classes.

 

[Wolfgang Grafen]

Hi again,

when I get far enough to parse the VHDL (which is not currently the
fact, but I have to look at the work coming up downstream) I will have
to put it into an internal data structure and then write some classes
to handle the MVC between whatever data I have and the PyQt4 widget
that is going to show the block diagram. I own the book "Rapig GUI
Programming with Python and Qt" by Mark Summerfield and try to read up
on the PyQt way of doing things as I try to make a plan for my
application. I have been looking for possible data structures with
google just to find out that most of the ideas are stored in
proceedings or to conferences or ieee papers not generally available
to me. Is anybody experienced with designing data structures willing
to share some ideas with me? Since I am using the open-source version
of PyQt4, any information will eventually be available to public (if
the app pass the planning stage) if that makes helping out any
easier: There is already an app called Qucs that is running in
Qt-3 and being ported to Qt-4 that is written in C++, but I don't know
how wise it is to just reverse-engineering C++ classes and translate
them into python classes.

For me it is not very clear what you intend to do. After years of
parsing parts of VHDL from time to time the rapid parsing way for me is
using regular expressions instead of one of the parser frame works
because of following reasons:

- It is hard for me to understand those frameworks
- They are very slow
- It is too much work for me to bring them up to work in a sensible way
- Compared with regular expression matching they usually need a lot of
extra work.

Regular expressions work very well once the validation of the VHDL code
was done by one of the commercial compilers.

Once you can rely on the code that it is valid VHDL
- Parse it with a regular expression for simple VHDL structures or
- Extract a structure first and analyse that with a set of regular
expressions

The result can be stored into a combination of lists and dictionaries,
dependend on the problem.

For processing with other tools the results could be stored into XML
structures.

PyQt as a widget framework is not useful until here, but of course you
could display your results in arbitrary graphical ways with PyQt, if you
rally need to. You should know, printing out an ASCII or XML
representation is so much more easy and quicker to code so I always
prefer that. There are even editors/visualizers ready to display XML...

Best regards

Wolfgang

 

[Svenn Are Bjerkem]

For me it is not very clear what you intend to do. After years of
parsing parts of VHDL from time to time the rapid parsing way for me is
using regular expressions instead of one of the parser frame works
because of following reasons:

- It is hard for me to understand those frameworks
- They are very slow
- It is too much work for me to bring them up to work in a sensible way
- Compared with regular expression matching they usually need a lot of
extra work.

I agree with frameworks being difficult to understand and that is why
I also have been using regular expressions in tcl to parse spice
netlists before. Now I want to parse spice, vhdl and also maybe
verilog. I think I will end up with regular expressions unless I get a
grip on SimpleParse.

The rationale for the whole project has been to finally be able to
view spice and specially vhdl code for projects I work on. This has
been something I have wanted to have for years, without having the
ressources to complete it. There are commercial tools available, but I
was looking for something more open/free that could be maintained
independently of what tools I have at work.

PyQt as a widget framework is not useful until here, but of course you
could display your results in arbitrary graphical ways with PyQt, if you
rally need to. You should know, printing out an ASCII or XML
representation is so much more easy and quicker to code so I always
prefer that. There are even editors/visualizers ready to display XML...

PyQt4 doesn't help me parse my sources, but it helps me visualise
them. I did something in tcl/tk to get hierarchical spice netlists
into a tree structure, but extending that app was too much hassle.
PyQt4 offers a lot of functionality once the threshold of learning it
has been passed. It also installs nicely on windows and most linux
distributions offer it ready to install. And I like Qt.

 

[Paddy]

Hi,
I am in the need to write an application for PyQt to visualise the
structure of a VHDL project I am working on. Looking for a sensible
way to parse VHDL files and putting them into a data structure that
PyQt can represent as a tree (or whatever the MVC is supporting)
through search engines does not give me many hints. From what I know,
VHDL is not a very easy language to parse. There seems to be a parser
for perl available, but I do not know if it is wise to use a perl
module as a template for writing something similar in python.

My initial idea is to start simple and extend features in my
application, but I fear that I may start off with wrong ideas how to
parse and then code myself into a dead-end requiring myself to rewrite
the whole application in order to get any further. I would start
finding definitions of entities and the instantiations of these and
build a tree from a set of external vhdl files stored in a file
hierarchy. If somebody have a starting point where to get going with a
task like this, I would be happy to know.

Been their, partially done that.
I started parsing an old version of VHDL (VHDL 87 I think); using
regular expressions and carrying state around myself. It was tough,
and I was doing it , but then Work mentioned that they were going to
need to parse later versions of VHDL and that was a whole new ball-
game, specifically configuration type information could be mixed in
with the architecture. Luckily work had also decided to by in a
commercial parser.

I learned that add-hoc parsers have limits to their maintainability
and become complex. You might not have such problems if your VHDL is
all in one library.

- Paddy.

 

[Henrique Dante de Almeida]

Hi again,

when I get far enough to parse the VHDL (which is not currently the
fact, but I have to look at the work coming up downstream) I will have
to put it into an internal data structure and then write some classes
to handle the MVC between whatever data I have and the PyQt4 widget
that is going to show the block diagram. I own the book "Rapig GUI
Programming with Python and Qt" by Mark Summerfield and try to read up
on the PyQt way of doing things as I try to make a plan for my
application. I have been looking for possible data structures with
google just to find out that most of the ideas are stored in
proceedings or to conferences or ieee papers not generally available
to me. Is anybody experienced with designing data structures willing
to share some ideas with me? Since I am using the open-source version
of PyQt4, any information will eventually be available to public (if
the app pass the planning stage) if that makes helping out any
easier: There is already an app called Qucs that is running in
Qt-3 and being ported to Qt-4 that is written in C++, but I don't know
how wise it is to just reverse-engineering C++ classes and translate
them into python classes.

Don't mix the parsing code with Qt. They're independent. Qt should be
used mostly for the model and the view. The controller should be
completelly decoupled from the model and the view. If you need to wrap
the controller in some Qt object, first write the controller without
Qt, then wrap it in the Qt object. Considering that you need a full
parser for VHDL, my option would be to take the VHDL grammar:

http://tams-www.informatik.uni-hamburg.de/vhdl/tools/grammar/vhdl93-bnf.html

and port it to a LR(1) or LALR parser that emits python code. See:

http://en.wikipedia.org/wiki/Comparison_of_parser_generators

Then, add action code to the grammar that you wrote, so that it
builds a tree representing the parsed file.

 

[Henrique Dante de Almeida]

 Don't mix the parsing code with Qt. They're independent. Qt should be
used mostly for the model and the view. The controller should be
completelly decoupled from the model and the view. If you need to wrap
the controller in some Qt object, first write the controller without
Qt, then wrap it in the Qt object. Considering that you need a full
parser for VHDL, my option would be to take the VHDL grammar:

 http://tams-www.informatik.uni-hamburg.de/vhdl/tools/grammar/vhdl93-b....

 and port it to a LR(1) or LALR parser that emits python code. See:

 http://en.wikipedia.org/wiki/Comparison_of_parser_generators

 Then, add action code to the grammar that you wrote, so that it
builds a tree representing the parsed file.

I think SableCC seems to be the best choice:

http://www.mare.ee/indrek/sablecc/