Continous Integration: Generating images from Gerbers

Hey,

for a while now I am looking for a way to more or less automatically generate images from the horizon gerbers once I export a new fabrication output. I tried python pcb-tools, but they are really not well documented and the holes ended upin the wrong place. Additionally I tried gerbv’s cli option, but lost patience as well.

Is there maybe a way to export PCB-images directly from horizon/cli? Maybe with the python module?

Which kind of image are you looking for? For once, we have the pdf export that can be converted to bitmap images using external tools. We also have the 3D render export that can be tuned to look 2D-ish by using orthographic projection.

Orthographic projections would be very cool. Think of it like this: imagine a open hardware project and there should be images in the Readme that represent the current state of the board and maybe a perspective view with parts on it or so. Currently I do this manually by creating screenshots.

Ideally I could write a pre-commit hook that does this whenever the board changes. Maybe this could even automatically export PDFs for the schematic, Fabrication output and BOMs.

I would maybe also write a blog post about this, because I think if this can be streamlined that would be a major thing for some people.

Would this be possible with the horizon module in python

All of this is possible with the python module / docker image right now. Unfortunately, the docs on the python module get rendered incompletely: https://github.com/horizon-eda/horizon-docs/blob/master/python.rst#3d-rendering-usage

@lukas Okay I still have issues with getting the Python module to run.

I have libpython-dev and python3-dev installed (Ubuntu 20.01), but I stil lget the following error when building using make build/horizon.so:

Package py3cairo was not found in the pkg-config search path.
Perhaps you should add the directory containing `py3cairo.pc'
to the PKG_CONFIG_PATH environment variable
No package 'py3cairo' found
 build/picobj/src/python_module/horizonmodule.o
src/python_module/horizonmodule.cpp:1:10: fatal error: Python.h: No such file or directory
    1 | #include <Python.h>
      |          ^~~~~~~~~~
compilation terminated.
make: *** [Makefile:859: build/picobj/src/python_module/horizonmodule.o] Error 1

Weirdly enough I have the cairo package installed:

Python 3.8.2 (default, Jul 16 2020, 14:00:26)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cairo
>>> cairo.cairo_version_string()
'1.16.0'
>>>

Yo’re probably missing python3-cairo-dev

Okay now I got rid of this error, and it compiles for a few minutes but fails at another point:

 build/picobj/src/export_3d_image/export_3d_image.o
src/export_3d_image/export_3d_image.cpp:3:10: fatal error: GL/osmesa.h: No such file or directory
    3 | #include "GL/osmesa.h"
      |          ^~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:859: build/picobj/src/export_3d_image/export_3d_image.o] Error 1

Any ideas?

You’re missing the osmesa development package. Install all packages listed in https://github.com/horizon-eda/horizon/blob/master/Dockerfile#L5

Okay that made it work : )

How would I import the horizon package then? I am a bit confused by the folder structure…

Put the horizonmodule.so in python’s sys.path. Either do this by appending the directory it’s in to sys.path or copying the module to one of the existing locations.

So I thought, but that file doesn’t exist…? I ran

git pull
make build/horizon.so

fzf says there is no file with that name in the tree:
Screenshot_20200803_090700

My bad, the module is called horizon.so.

Success

>>> import sys
>>> sys.path.append("/home/d0/src/horizon/build")
>>> import horizon
>>> horizon.__dir__()
['__name__', '__doc__', '__package__', '__loader__', '__spec__', 'Project', 'PoolManager', 'Pool', '__file__']

Thanks for the guidance. Are there plans to integrate this python functionality with official builds or with pip – so essentially make the whole thing a little bit more straightforward to install?

Edit: I ask because it would make sense to document the whole thing a little better – but if there are plans maybe that would be unnecessary legwork.

Edit 2: Now that I thought a little bit about this, maybe it would be even better to make this part of the CLI functionality, as this feels to me more like the place where something like this should be.

So a bit like:

# Export BOM
horizon-eda schematic export bom -i myproject.hprj -o bom.csv

# Export Schematic PDF
horizon-eda schematic export pdf -i myproject.hprj -o schematic.pdf

# Export Gerber
horizon-eda board export gerber --zip -i myproject.hprj -o myproject.zip

# Export 3D-Views
horizon-eda board export 3D --top --orthographic --no-parts -i myproject.hprj -o top.png
horizon-eda board export 3D --bottom --orthographic --no-parts -i myproject.hprj -o bottom.png
horizon-eda board export 3D --angle [60,60,0]  -i myproject.hprj -o perspective.png

# etc

What do you think?

Which official buillds are you referring to? So far there aren’t any where integrating the python module would make sense. As for pip, I don’t really see which problem that’s supposed to solve since the setup.py will be no more than a wrapper around the existing Makefile. Python’s dependency management also won’t be of much use since all dependencies are shared libraries rather than other python modules.

Definitely: as far as I can tell, a list of dependencies and perhaps a bit more words on how to import the module once built would help.

I think that the python module is a good foundation for such CLI. There were several reasons for exposing this feature set as python module rather than a CLI application

Spawning processes can be slow and everything needs to be stuffed into the arguments. With an interface such as the one you proposed, the project needs to be loaded over and over again. While this may no big deal for exporting gerbers, it would make https://blog.horizon-eda.org/progress/2020/04/28/progress-04.html#exporting-3d-renderings unusably slow. Also passing arguments as dicts comes for free since the json serialization is already there for saving. Another approach would have been to have some kind of interactive horizon-sh shell that’s a TCL or Lua interpreter that interfaces to the horizon bits. All things considered, the python module is a good sweet spot between features and ease of use.

Okay I got the reasoning behind the decision now. What I tried to allude to in the other points (pip, including in builds foo), is that I think getting started with this should be as straightforward as possible or feasible. Right now this requires compiling horizon yourself, but what about releases and packages (e.g. via apt)?

Ideally I’d imagine something like:

  1. Install a horizon release or package
    1.1 Maybe ask about whether to install the python module
  2. Start python and import horizon

The pip idea was more among the lines of being able to pip install horizon-eda and then beeing able to use it, but I can see where this might get hairy (how to maintain version compatibility between the horizon binary and the python module etc).

It’s just that building quality debian packages is a dark art that I still don’t understand. Debian people already package the app, so their work could be used as a starting point for project-provided binaries.

Perhaps that’s worth opening an issue hoping that other people will jump in.

EDIT: As for the python module, there’s definitely scope for improvement: Right now, the python module is built using the makefile https://github.com/horizon-eda/horizon/blob/master/Makefile#L839 as that was the easiest way. This leaves us without a proper way to install the module into the right directory.

Okay gotcha, that is why I was asking about the pip thing, because that is where I have a little more experience. I now opened an issue on this, where discussion can continue.