OVC Mini Network Tester and Conversion PCB #73

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open

gbalke wants to merge 40 commits into osrf:master from gbalke:ovc_mini

README.md

-Original file line number
+Diff line change
@@ Expand Up @@
      * ovc3: three ON Semi AR0144CS imagers, USB Type-C peripheral, Trenz TE0820 module with Xilinx Zynq UltraScale+, RAM, flash, etc. Up to four optional external camera boards, each of which add a pair of AR0144CS imagers.
      * ovc4: A Jetson Xavier NX carrier board with an NXP MCU, USB Type-C interface and six Picam compatible connectors.
      * ovc5 (current work): based on Zynq UltraScale+ modules from Enclustra, with up to six external MIPI camera boards. Both Zynq USB SuperSpeed (5 Gbps) transceivers connect to an onboard USB-SS+ (10 Gbps) hub controller IC, which multiplexes them upstream via a USB type-C connector. Also has three Qwiic/STEMMA connectors and some random FPGA GPIO for low-speed expansion.
+     * ovc-mini (current work): based on the Ka-Ro QSXM/QSXP modules, this project is attempting to push the limits of the modular camera system in terms of size. The module will have the same footprint as one of the MIPI camera modules (36x36mm) and be able to stream a single imager's output up to the limits of the IMX8M(M/P) chipset's capabilities.
     # Where do I find stuff
@@ Expand Down @@

ovc_mini/README.md

-Original file line number
+Diff line change
@@ -0,0 +1,31 @@
+    # OVC Mini
+    OVC Mini intends to bring the same open computing platform to a much smaller form factor.
+    This had been made possible through the extremely small form factor SoMs that are built
+    around NXP's IMX8M chips. The end goal is a minimalistic footprint camera with the same
+    screw-on lens, combined with swappable 2-lane or 4-lane mipi camera boards.
+    ## Hardware
+    Ka-Ro's QSXP/QSXM SoMs are perfectly sized for the pre-existing OVC-lens holder footprint.
+    Using these modules, it will be possible to create a sandwich of the imager board with the
+    compute board for minimal wasted space.
+    ## Firmware
+    This project is running Ka-Ro's custom version of Yocto Hardknott with some additional drivers
+    added in for camera support.
+    ## Benchmark
+    There are two benchmarks of interest:
+    * Latency between the image being read from the sensor to arriving in the host computer
+    * Reliable transmission bandwidth
+    Tools to test these are located in the benchmark directory.
+    ## Software
+    The goal is to re-use much of OVC5's software stack by using udmabufs again for the output of the camera modules. I2C is already the same. This will allow for a simple change of the [yaml configuration file](https://github.com/osrf/ovc/blob/master/ovc5/software/config.yaml) to switch between OVC5 and OVC Mini.
+    __TODO:__ bring out the gpio settings for line trigger and sample trigger into the configuration file.

ovc_mini/benchmark/.gitignore

-Original file line number
+Diff line change
@@ -0,0 +1,3 @@
+    output/
+    build/
+    __pycache__/

ovc_mini/benchmark/README.md

-Original file line number
+Diff line change
@@ -0,0 +1,30 @@
+    # Network Bandwidth and Reliability Testing
+    __NOTE__: Both client and server must have iperf3 installed before continuing.
+    These scripts are meant to determine the maximum usable bandwidth over a
+    network connection for a variety of packet sizes. To begin, connect the
+    device to be tested to the same network as your host machine. Next, run the
+    following command:
+    ```shell
+    ./test.py username ip_address
+    ```
+    The test should now run for some time. A help menu is available for more
+    detailed control of the test (`./test.py -h`).
+    After the test is concluded, a folder labeled `output` should be in the host's
+    current directory. Finally, run the processing script to generate some nice
+    graphs to help visualize the data!
+    ```shell
+    ./process.py
+    ```
+    The graphs generate one at a time for results of a given test. Close a graph
+    to see results of the next test.
+    ## Reference Material
+    MCS Table: https://semfionetworks.com/blog/mcs-table-updated-with-80211ax-data-rates/

ovc_mini/benchmark/benchmark/Makefile

-Original file line number
+Diff line change
@@ -0,0 +1,10 @@
+    CC=g++
+    ARM_CC=aarch64-linux-gnu-g++
+    local:
+    	mkdir -p build
+    	${ARM_CC} src/test_server.cpp -lpthread -o build/test_server
+    arm:
+    	mkdir -p build
+    	${ARM_CC} src/test_server.cpp -lpthread -o build/test_server_arm

ovc_mini/benchmark/benchmark/__init__.py

-Original file line number
+Diff line change
@@ -0,0 +1,4 @@
+    """Benchmark init"""
+    from benchmark.ssh_session import SSHSession
+    from benchmark.base import Benchmark, IperfBench, PayloadBench

ovc_mini/benchmark/benchmark/base.py

-Original file line number
+Diff line change
@@ -0,0 +1,195 @@
+    """Defines the base test class"""
+    import numpy as np
+    import dataclasses
+    import json
+    import os
+    import socket
+    import subprocess
+    import time
+    from typing import Any, Dict, List
+    from pathlib import Path
+    from benchmark.ssh_session import SSHSession
+    FILE_DIR = Path(os.path.dirname(os.path.abspath(__file__)))
+    # Number of times to retry an unreliable execution before erroring.
+    RETRY_COUNT = 5
+    class Benchmark:
+        """Base class for benchmark collection."""
+        @dataclasses.dataclass
+        class Results:
+            """Stores data related to general test results."""
+            duration: float  # seconds
+            interval: float  # seconds
+            sample_count: int
+        def __init__(self, session: SSHSession, address: str, port: int):
+            self._session = session
+            self._address = address
+            self._port = port
+    class IperfBench(Benchmark):
+        """Manages an iperf3 test on the remote server."""
+        @dataclasses.dataclass
+        class Results(Benchmark.Results):
+            """Stores data related to iperf test results."""
+            window_size: int  # Bytes
+            parallel_streams: int  # Number of threads sending data
+            average_bandwidth: float  # Mbps
+            sigma: float  # Mbps
+            ci_2: float  # Mbps 95% confidence interval
+            ci_3: float  # Mbps 99.7% confidence interval
+        def __enter__(self):
+            self._session.run(f"iperf3 -s -p {self._port} > /dev/null 2>&1 &")
+            return self
+        def __exit__(self, exit_type, value, traceback):
+            # Close all iperf3 sessions
+            self._session.run(f"killall iperf3")
+        def run_test(self, duration: float, interval: float, parallel_streams: int,
+                     window_size: int) -> Results:
+            # Read https://www.cisco.com/c/en/us/support/docs/wireless-mobility/wireless-lan-wlan/212892-802-11ac-wireless-throughput-testing-and.html
+            cmd = (
+                f"iperf3"
+                f" -R"  # Reverses iperf so the server sends to the client
+                f" -J"  # Output in JSON format
+                f" -c {self._address}"
+                f" -p {self._port}"
+                f" -i {interval}"
+                f" -t {duration}"
+                f" -P {parallel_streams}"
+                f" -w {window_size}k")
+            out = None
+            retry = 0
+            while out is None:
+                try:
+                    out = subprocess.check_output(cmd, shell=True)
+                except subprocess.CalledProcessError as e:
+                    if retry > RETRY_COUNT:
+                        raise e
+                    retry += 1
+                    time.sleep(1)
+            return self.Results(interval=interval,
+                                duration=duration,
+                                window_size=window_size,
+                                parallel_streams=parallel_streams,
+                                **self.process_results(out))
+        @staticmethod
+        def process_results(iperf_output: str) -> Dict[str, Any]:
+            """Processes iperf output."""
+            def to_Mbps(bps: float) -> float:
+                return round(bps * 1e-6, 2)
+            data = json.loads(iperf_output)
+            bps = []
+            for interval in data['intervals']:
+                bps.append(interval['sum']['bits_per_second'])
+            samples = len(data['intervals'])
+            avg = np.average(bps)
+            sigma = np.std(bps)
+            ci_2 = max(0, avg - (2 * sigma))
+            ci_3 = max(0, avg - (3 * sigma))
+            return {
+                'sample_count': samples,
+                'average_bandwidth': to_Mbps(avg),
+                'sigma': to_Mbps(sigma),
+                'ci_2': to_Mbps(ci_2),
+                'ci_3': to_Mbps(ci_3),
+            }
+    class PayloadBench(Benchmark):
+        """
+        Run simulated load test.
+        This is defined as a fixed size packet queued at a fixed interval.
+        We are interested if the packets get backed up (meaning bandwidth is
+        not able to keep up with the packet queue).
+        """
+        TEST_PROGRAM_NAME = 'test_server_arm'
+        @dataclasses.dataclass
+        class Results(Benchmark.Results):
+            """
+            Stores data related to iperf test results.
+            Attributes:
+                start_time: time that the test started. Does not have to be system
+                    time.
+                packet_received: time a packet was received from start of test.
+                    The data should only be relative to the start time.
+            """
+            packet_size: int  # Megabytes
+            start_time: float
+            receive_times: List[float]
+        def __enter__(self):
+            subprocess.check_output(f"cd {FILE_DIR} && make arm", shell=True)
+            self._session.scp(FILE_DIR / 'build' / self.TEST_PROGRAM_NAME,
+                              Path('/tmp'))
+            return self
+        def __exit__(self, exit_type, value, traceback):
+            self._session.run(
+                f"if [[ ! -z $("
+                f"ps aux | grep {self.TEST_PROGRAM_NAME} | grep -v grep)"
+                f" ]]; then"
+                f" killall {self.TEST_PROGRAM_NAME};"
+                f" fi;")
+            self._session.run(f"rm /tmp/{self.TEST_PROGRAM_NAME}")
+        def run_test(self, packet_size: float, duration: int, interval: float):
+            # Read https://stackoverflow.com/a/29172 to understand backgrounding
+            # in an ssh command.
+            cmd = (f"nohup /tmp/{self.TEST_PROGRAM_NAME}"
+                   f" -s {packet_size}"
+                   f" -p {self._port}"
+                   f" -d {duration}"
+                   f" -i {interval}"
+                   f" > /dev/null 2>&1 &")
+            self._session.run(cmd)
+            packet_bytes = packet_size * 1000000
+            time.sleep(1)
+            with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
+                connected = False
+                while not connected:
+                    try:
+                        s.connect((self._address, self._port))
+                        connected = True
+                    except ConnectionRefusedError as e:
+                        print(e)
+                        time.sleep(1)
+                data_count = 0
+                start = time.time()
+                now = start
+                packet_count = 0
+                receive_times = []
+                while (now - start < duration):
+                    if data_count >= packet_bytes:
+                        receive_times.append(now)
+                        packet_count += 1
+                        data_count -= packet_bytes
+                    data_count += len(s.recv(4096))
+                    now = time.time()
+                total_time = now - start
+                return self.Results(packet_size=packet_size,
+                                    interval=interval,
+                                    duration=total_time,
+                                    sample_count=packet_count,
+                                    start_time=start,
+                                    receive_times=receive_times)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OVC Mini Network Tester and Conversion PCB #73

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!

OVC Mini Network Tester and Conversion PCB #73

Are you sure you want to change the base?

Uh oh!

OVC Mini Network Tester and Conversion PCB #73

Uh oh!

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Uh oh!