3. Test and validate your component library
As your team’s library of devices and circuits evolves and is validated, it is important to maintain the quality of the library. To that end, you can add regression tests to your library. With Luceda IP Manager, you can test the layout, netlist and simulation model of the devices and circuits in your team’s library for changes. The devices and circuits can be tested against known-good reference files. Other tests can be added as well, such as detailed tests of the circuit model.
Such tests should be run regularly. They should definitely be run whenever you make any change to the library. Luceda IP Manager uses the standard Python test framework pytest which provides an excellent platform to make this part of your team’s standard workflow.
Changes can be desired or undesired:
A desired change can be, for instance, a parameter update of a device after testing reveals a value for that parameter that gives more optimal performance. After making the change, the tests will catch such a change, and subsequently, the test itself can be updated to reflect the new desired state.
An undesired change can, for instance, be an unintended change in an algorithm inside a PCell in the library. The tests will fail after such a change is made. The author of the algorithm can then go back and fix the algorithm, after which the tests should pass again.
Changes can also originate from elsewhere. In particular, the PDK on which your team’s library is based may have been changed with a difference in layout or models of devices in your library as a result. By applying and consistently running regression tests on your library, you can therefore significantly reduce the risk for errors and increase the quality of your team’s library.
In this tutorial, we show how to equip the library pteam_library_si_fab that was developed in Develop and distribute your component library with tests based on Luceda IP Manager.
Additional resources
The Luceda IP Manager documentation goes in more detail about the concepts and features of IP Manager.
The Luceda IP Manager tutorial discusses in depth how to add regression tests to a library or PDK.
3.1. Configuring the library for testing
In order to add tests to pteam_library_sifab, we first extend its folder structure with a tests folder with a few essential files:
pteam_library_si_fab
├── building
│ └── build_pteam_library_si_fab.py
├── ipkiss
│ └── pteam_library_si_fab
│ ├── components
│ ├── technology
│ ├── __init__.py
│ └── all.py
└── tests
├── conftest.py
├── generate_reffiles.py
└── pytest.ini
conftest.pycontains predefined data that can be used by each test.generate_reffiles.pyis a script to generate the known-good reference files for the tested components.pytest.inicontains configuration settings for pytest.
In conftest.py, we specify the PDK technology to be loaded for testing.
Our library is based on si_fab so we use the si_fab PDK technology:
# pytest configuration file
import pytest
# import the technology used for testing this library for the full test session
@pytest.fixture(scope="session", autouse=True)
def TECH():
from si_fab.all import TECH
return TECH
In pytest.ini, we specify that we want to have an HTML report.
The pytest-html plugin of pytest is used for that purpose.
This all comes pre-installed with Luceda IPKISS.
[pytest]
addopts = '--html=report.html' '--self-contained-html'
Finally, generate_reffiles.py contains a single function call to generate the known-good reference files for the tests that we will add in the next step.
# Regenerates the reference files for the tests in this directory.
# Be careful when using this function: it will overwrite all reference files if they already exist.
# When the reference files already exist, and tests are failing, make sure to compare the generated files
# against the reference files to spot for (un)expected changes.
from ip_manager.testing import generate_reference_files
generate_reference_files()
3.2. Adding component reference tests to the library
Now that the basic configuration has been made, we can start to equip our library with tests. We will add regression tests of the LayoutView, NetlistView and CircuitModelView of the Mux2Heated and SplitterTree components in pteam_library_si_fab.
Tests are defined in Python script files that have a name starting with test_. Pytest will automatically recognize those and scan them for tests that need to be run.
It is advised to organize the tests for different components or families of components in separate test files. In this case, we test the lattice filter and the splitter tree in separate files:
pteam_library_si_fab
├── building
│ └── build_pteam_library_si_fab.py
├── ipkiss
│ └── pteam_library_si_fab
│ ├── components
│ ├── technology
│ ├── __init__.py
│ └── all.py
└── tests
├── conftest.py
├── generate_reffiles.py
├── pytest.ini
├── test_mux2.py
└── test_splittertree.py
In test_mux2.py, we define a test for the Mux2Heated PCell that will test several aspects of the cell against known-good reference files:
from ip_manager.testing import Compare, ComponentReferenceTest
import pytest
import numpy
@pytest.mark.comparisons([Compare.LayoutToGds, Compare.LayoutToXML, Compare.NetlistToXML, Compare.SMatrix])
class Test_Mux2Heated(ComponentReferenceTest):
@pytest.fixture
def component(self):
from pteam_library_si_fab.all import Mux2Heated
return Mux2Heated(name="Mux2HeatedDefault")
@pytest.fixture
def wavelengths(self):
return numpy.linspace(1.54, 1.56, 200)
Let’s analyze this script:
First, we import
ComponentReferenceTestandComparefrom theip_manager.testingpackage.ComponentReferenceTestis a class that we will use to define a test based on known-good reference files for a component.Compareis a Python enumeration of possible comparisons.We also import the
pytestandnumpypackages.Then we define our test by means of a class that starts with
Test_. Pytest will automatically recognize this as a test. By inheriting fromComponentReferenceTest, we get all the test functionality provided by Luceda IP Manager.The class is marked with the tests that we want to run using
@pytest.mark.comparisonswhich takes a list ofCompareitems:Compare.LayoutToGdscompares the LayoutView to a known-good GDSII file. The benefit of this is that GDSII is what you will use for tape-out.Compare.LayoutToXMLcompares the LayoutView to a known-good XML file. The benefit of this is that it includes detailed information about the optical and electrical ports.Compare.NetlistToXMLcompares the NetlistView to a known-good XML file. This way you can be sure that the logical interface of your component is correctly defined as well.Compare.SMatrixcompares the S-parameter matrix generated by the CircuitModelView to a known-good S-matrix data file.
Inside the class we define two methods. Both methods are decorated with
@pytest.fixturewhich instructs pytest that this is data that needs to be generated before running the test.component(self)defines the Luceda IPKISS PCell object which we want to test. In this case, we loadMux2Heatedfrompteam_library_si_fab.alland instantiate it. Note that we assign a specific name to the component. IP Manager tests should not rely on the automatic cell name generation of IPKISS, because pytest tests are not guaranteed to run in a specific order. When relying on automatically generated names, tests could fail because of differences in test execution order.wavelengths(self)returns an array of wavelengths for whichCompare.SMatrixwill calculate and compare the S-matrix.
The test for SplitterTree in test_splittertree.py follows exactly the same pattern.
3.3. Generating and checking the known-good reference files
Before we can run these tests, we’ll need to obtain the known-good reference files first and check that they are correct.
From your IDE (PyCharm), simply execute the generate_reffiles.py script.
The result is a new subfolder ref which contains 4 files for each of the components corresponding to the 4 comparisons defined above.
pteam_library_si_fab
├── building
├── ipkiss
└── tests
└── ref
├── Test_Mux2Heated_caphe.smat
├── Test_Mux2Heated_layout.xml
├── Test_Mux2Heated_layout_nofilter.gds
├── Test_Mux2Heated_netlist.xml
├── Test_SplitterTree_caphe.smat
├── Test_SplitterTree_layout.xml
├── Test_SplitterTree_layout_nofilter.gds
└── Test_SplitterTree_netlist.xml
If you open the _layout.xml file, you can see that it contains a full hierarchical representation of the layout in XML format and check that the properties of the ports are correct:
<IPKISSLIBRARY name="MUX2HEATED_1">
<STRUCTURE name="MUX2HEATED_1">
<ELECTRICALPORT layer="PPLAYER M1-DRW" name="ht_in0" position="(14.520, 1.325)" />
<ELECTRICALPORT layer="PPLAYER M1-DRW" name="ht_out1" position="(96.576, 1.325)" />
<ELECTRICALPORT layer="PPLAYER M1-DRW" name="ht_out3" position="(207.419, -1.325)" />
<PORT angle="0.00" name="out1" position="(227.069, -1.325)" trace_template="SI_FAB_WIRE_WGTEMPLATE_1" />
<ELECTRICALPORT layer="PPLAYER M1-DRW" name="ht_in3" position="(177.419, -1.325)" />
<ELECTRICALPORT layer="PPLAYER M1-DRW" name="ht_in2" position="(118.379, -1.325)" />
<ELECTRICALPORT layer="PPLAYER M1-DRW" name="ht_out0" position="(44.520, 1.325)" />
<PORT angle="180.00" name="in1" position="(-9.520, -1.325)" trace_template="SI_FAB_WIRE_WGTEMPLATE_3" />
If you open the _layout_nofilter.gds file in KLayout, you can check in detail whether the component is correctly implemented:
Known-good GDSII reference file for Mux2Heated.
3.4. Running the tests
Once the reference files are generated, we are ready to run the tests for the first time. In order to conveniently do that from PyCharm, first make sure that pytest is used as the default test runner. In the File>Settings menu, go to Tools and then Python Integrated Tools. In the Default test runner field, select pytest.
Set pytest as the default test runner in PyCharm.
The tests can now be run from PyCharm by right-clicking on the test_mux2.py file and selecting Run `pytest in test_mux2..`.
PyCharm shows the test results in a separate pane:
Test pane in PyCharm with test results.
You can also run tests for the whole library by selecting the tests folder and selecting `` Run pytest in tests`` from the context menu.
3.5. Updating tests after a desired change
Imagine that after experimental testing of the Mux2Heated component, it turns out that it works better with a slightly different first power coupling parameter:
power_couplings updated from 0.5 to 0.49. class Mux2Heated(MZILatticeFilterHeated):
"""Two-way wavelength demultiplexer with a passband of 50% of the FSR at the center_wavelength.
"""
_name_prefix = "MUX2HEATED"
power_couplings = i3.LockedProperty(default=[0.48, 0.13, 0.12, 0.5, 0.25])
...
When we run the tests after making this change, we see that 3 out of the 4 tests fail:
Tests fail after a known change.
The netlist test is still passing since we did not change that, but the layout and S-matrix have changed due to the parameter change. Hence, the new layout and model are not equal to the reference files anymore.
For each test, a temporary file was saved in the same ref folder, which reflects the new version.
These temporary files have almost the same name but an extra .tmp extension is added:
Temporary (new) files saved along the reference files.
We can now, for instance, overlay the two GDSII files in KLayout and check that the .tmp.gds file correctly reflects the new state:
Overlay of the reference and temporary GDSII file (coloring of layers changed to see the difference).
After you validate that the .tmp file is the new known-good file, just replace the old reference file with the .tmp file by removing the old file and renaming the .tmp file.
3.6. Where to go from here
The tests that we have added here test the components with their default parameters and only test the two high-level PCells in the library. Tests can be extended to cover also the other supporting code in pteam_library_si_fab or to test the PCells across a range of parameters. See Getting started: increasing test coverage and Advanced features and options for in-depth guidelines.