Getting started: increasing test coverage

In this section we will add additional tests to our library, so that we cover more views and behavioral aspects of the cells. The higher the test coverage, the higher the chance to detect errors early on.

Add a circuit model reference test

The SMatrix reference test Compare.SMatrix compares the S-matrices generated by the component’s circuit model to those saved in a reference file. Adding it is done by adding Compare.SMatrix to the comparisons mark and generating the reference file:

from ip_manager.testing import ComponentReferenceTest, Compare
import pytest
import numpy as np


@pytest.mark.comparisons([Compare.LayoutToGds, Compare.LayoutToXML, Compare.NetlistToXML, Compare.SMatrix])
class TestDiskResonator(ComponentReferenceTest):
    @pytest.fixture
    def component(self):
        # Create and return a basic disk resonator.
        from my_library.all import DiskResonator
        my_disk = DiskResonator(name='my_disk')
        my_disk.Layout(disk_radius=10.0)
        return my_disk

    @pytest.fixture
    def wavelengths(self):
        return np.arange(1.3, 1.8, 0.0001)

  • The PCell will need a CircuitModelView definition respectively, if not the test will fail.

  • This S-model is calculated over a specific wavelength range that can be configured by adding the wavelengths fixture to the TestDiskResonator class. The default wavelength range is 1.4 to 1.7 micrometres.

A small tolerance is allowed before two S-models are considered unequal, see S-model testing for more information.

In order to generate the reference file for this additional test only, we can rerun generate_reference_files(path, match_tests) using the match_tests argument to only process tests that have ‘smatrix’ in their name:

from ip_manager.testing import generate_reference_files
generate_reference_files(path="", match_tests="smatrix")

Alternatively, you can run pytest with the -k argument. (For regenerating reference files, see Generating the known-good reference files)

pytest tests --regenerate -k smatrix

This will only process the tests that have smatrix in the name. Users can pass the -k argument to pytest to filter tests in a flexible way.

Add a layout-versus-netlist test

IP Manager can use the IPKISS netlist extraction features to perform a type of layout-versus-schematic (LVS) or rather layout-versus-netlist comparison: it will extract a netlist from the LayoutView (ports, instances of other cells and their connectivity) and compare it with the NetlistView of the component. (Note this is mostly useful for components or circuits which do not use i3.NetlistFromLayout).

If the netlist extraction raises warnings or errors e.g. because of bad connections due to mismatched waveguides, you will get these for free as well!

Implementing this test is done by inheriting from ip_manager.testing.LayoutToNetlistTest:

# extra import
from ip_manager.testing import LayoutToNetlistTest

# test class inheriting ``LayoutToNetlistTest``
 class TestDiskResonatorLVS(LayoutToNetlistTest):
     @pytest.fixture
     def component(self):
         # Create and return a basic disk resonator.
         from team_library.all import DiskResonator
         my_disk = DiskResonator(name='my_disk')
         my_disk.Layout(disk_radius=11.0)
         return my_disk

Sharing comparisons to run across tests

If multiple components share the same set of tests, we can define one variable to specify the tests:

from ip_manager.testing import ComponentReferenceTest, Compare
import pytest

comparisons = [Compare.LayoutToGds, Compare.LayoutToXML]

@pytest.mark.comparisons(comparisons)
class TestDiskResonator(ComponentReferenceTest):
    @pytest.fixture
    def component(self):
        # Create and return a basic disk resonator.
        from diskres_cell import DiskResonator
        my_disk = DiskResonator(name='my_disk')
        my_disk.Layout(disk_radius=10.0)
        return my_disk

@pytest.mark.comparisons(comparisons)
class TestDiskResonator2(ComponentReferenceTest):
    @pytest.fixture
    def component(self):
        # Create and return a basic disk resonator.
        from diskres_cell import DiskResonator
        my_disk = DiskResonator(name='my_disk')
        my_disk.Layout(disk_radius=15.0)
        return my_disk

Add your own circuit checks

IP Manager also allows you to define custom circuit model tests on the S-parameters.

This is done by inheriting from CircuitModelTest and implementing a custom test method that creates an SMatrixTester object.

This can be combined in the same class as the component reference tests but could also be done in a separate test class.

See the full code below for an example:

from ip_manager.testing import ComponentReferenceTest, CircuitModelTest, SMatrixTester, Compare

import numpy as np
import pytest


@pytest.mark.comparisons([Compare.NetlistToXML, Compare.LayoutToGds, Compare.SMatrix])
class TestDiskResonatorPeaks(ComponentReferenceTest, CircuitModelTest):

    @pytest.fixture
    def component(self):
        # Create and return a basic disk resonator.
        from team_library.all import DiskResonator
        my_disk = DiskResonator(name='my_disk')
        my_disk.Layout(disk_radius=10.0)
        my_disk.CircuitModel(disk_n_eff=3.0)
        return my_disk

    # Used by circuit model tests (optional)
    @pytest.fixture
    def wavelengths(self):
        return np.arange(1.3, 1.8, 0.0001)

    # You can specify your own tests in this method.
    # `smatrix` is a fixture provided by `CircuitModelTest` so you don't need to care about calculating it yourself.
    def test_disk_smatrix(self, smatrix):

        # Create a tester object
        tester = SMatrixTester(smatrix, central_wl=1.55)

        # Perform general tests
        assert tester.is_reciprocal(), "Component is not reciprocal"
        assert tester.is_passive(), "Component is active"

        # Select a terminal link
        tester.select_ports('in', 'out')  # from 'in' to 'out'

        # Perform tests on this link
        assert tester.has_peak_at(centre=1.5, rtol_centre=0.001, test_fwhm=False), "No peak at 1.5 um"
        assert tester.has_peak_at(centre=1.579, rtol_centre=0.001, fwhm=0.004, rtol_fwhm=0.1), "No peak at 1.579 um, or wrong FWHM"
        assert tester.fsr_isclose(0.08, rtol=0.01), "Free spectral range is wrong"
        assert tester.il < 0.1, "Insertion loss is too high"

Standard python assert statements are used. These will raise an error and make the test fail when the statement after assert is not True. Pytest has several functions available for testing that you can use as well, such as pytest.approx.

Try changing the disk_n_eff value and you’ll see that the test fails when its value differs from 3.0. If HTML reporting (see HTML output) is enabled, a plot of the S-matrix will be automatically inserted for easy inspection. In this case, the peaks would visibly shift away from 1.5 and 1.579 micrometres.

A complete example file can be downloaded here: test_diskresonator_smatrix.py. Note that, in this file, we added a test that fails in order to simulate a situation in which modifications resulted in a regression.

Add sample tests

Including small sample files is a valuable practice for illustrating component usage or showcasing circuit behaviors within your library. These files serve as useful starting points for users and also aid in documentation and training. However, it’s important to note that as component implementations evolve, their interfaces may change, potentially breaking the sample code.

To mitigate this, it’s advisable to incorporate tests that automatically verify the continued functionality of our sample files. Fortunately, setting up these tests is straightforward.

For instance, let’s assume we’ve added a docs folder to our library project, containing documentation and example files. Additionally, within the components folder, an example file demonstrating the usage of the DiskResonator PCell is included. This example file could contain the following code:

from team_library.all import DiskResonator

my_disk = DiskResonator(name="my_disk")

tunable_delay_lv = my_disk.Layout()
tunable_delay_lv.visualize()

To test these sample files, we’ll create a new test file named test_samples.py to configure the tests and add it to tests\.

Initial structure of the `team_library` project with samples and sample tests files added.

File structure of the team_library project with samples and sample tests files added.

Sample tests are automatically generated for all the sample files that are added to the configuration. To setup this configuration, a new test class is constructed that inherits from SampleTest. In this class, the root directory and a list of sample paths are defined. A sample path can either be the path to a specific file or to a directory. Given a directory, IP manager will automatically discover which files are sample files by checking the filename against a list of expressions, e.g. python files starting with example_ will be included. The execution of sample files can be further configured by providing a pytest mark together with the filename.

A sample test file could for example look like this:

from ip_manager.testing import SampleTest
from os.path import abspath, join, dirname, pardir
import pytest

class TestLibrarySamples(SampleTest):
    def work_dir(self):
        return abspath(join(dirname(__file__), pardir))

    def sample_src(self):
        return [
            "team_library/ipkiss/team_library/components/",
            ("docs/sample_circuit_simulation.py", pytest.mark.skip(reason="Long simulation")),
        ]

Running this test will result in the execution of every discovered sample file and the verification whether it can be executed without any errors. An exception thrown during the execution of a sample file will result in a failed test for that file.

A complete example test file can be downloaded here: test_samples.py.

Note

In the pytest window in PyCharm, only the test class will be listed, not the individually discovered test cases for each sample file. For a comprehensive overview of executed tests and their outcomes, consult the generated HTML test report.