All notable changes to the kytos project will be documented in this file.
Unhandled exception on a
listen_to
decorated function (running in a ThreadPool) is logged as error.New
--database
configuration option that supportsmongodb
MongoDB client for NApps,
Mongo
available onkytos.core.db
moduleAdded a wait mechanism during controller startup time to ensure the database is reachable if it's been configured
pymongo
andpydantic
(for database models) are now core dependenciesAdded MongoDB environment variables
MONGO_HOST_SEEDS, MONGO_USERNAME, MONGO_PASSWORD
Added optional MongoDB environment
MONGO_DBNAME, MONGO_MAX_POOLSIZE, MONGO_MIN_POOLSIZE, MONGO_TIMEOUTMS
Added a docker-compose.yml file for local development to compose with MongoDB replica set cluster
Added an in-memory dead letter structure for unhandled exceptions of KytosEvents indexed by their names
Added core endpoints for the dead letter structure:
GET /api/kytos/core/dead_letter/?event_name=<name> PATCH /api/kytos/core/dead_letter/ (requires request body) DELETE /api/kytos/core/dead_letter/ (requires request body)
Added
tenacity
as a core dependency for retries.New
--apm
configuration option that supportselasticsearch
APM (Application Performance Monitoring)kytosd
Elastic APM integration provides instrumentation for MongoDB, Flask, requests andKytosEvent
@begin_span
decorator for on-demand APM custom functions/methods instrumentationAugmented docker-compose.yml to also spin up Elastsearch, Kibana and APM server with authentication
Augmented docker-compose to also spin up Filebeat, integrating log file as input
The
listen_to
decorator now supports apool
keyword argument to specify which thread pool the execution should be submittedNew core
kytos.core.retry
module provides decorators for retries based ontenacity
Unhandled exceptions on
@listen_to
decorator now also include a tracebackAdded
@alisten_to
decorator forasync
methods. NApps can subscribe to events asynchronously with this decorator as needed.
Kytos controller can shutdown if the database is configured but not reachable during startup time.
Augmented
KytosEvent
with internal attributes (id
andreinjections
), no breaking changes.KytosEvent
now optionally supports atrace_parent
argument for APM distributed tracing to also instrument and correlateKytosEvent
.Added file formatter and file handler boilerplate on logging.ini.template to facilitate hooking the file handler on logger_root and logger_kytos as needed.
Broke compatibility in the
thread_pool_max_workers
, it uses a dict now instead of a single integer. If you were using a single integer for a global pool, please migrate it to{"sb": 256, "db": 256, "app": x}
, where x should be the value that you used to use or the default 512.The following pools are available by default to be used in the listen_to decorator with the
pool
option:sb: it's used automatically by kytos/of_core.* events, it's meant for southbound related messages app: it's meant for general NApps event, it's default pool if no other one has been specified db: it can be used by for higher priority db related tasks (need to be parametrized on decorator), it's also used automatically by kytos.storehouse.* events
msg_out
core queue now leverages a PriorityQueue instead of a FIFO Queue.msg_in
core queue now leverages a PriorityQueue instead of a FIFO Queue.
- Fixed file already exists error when creating config dirs, issue 222
- Load NApps ordered by modification, allowing the administrator to set a desired order of loading.
- New README reflecting the change to Kytos NG.
- Support python 3.9.
- Method to create or update interface.
- Run tests using GitHub Actions.
- Lock to avoid race conditions when selecting a tag.
- Lock to avoid race conditions when getting or creating a Switch.
- New blueprint: EP023 - Kytos Pathfinder Filter Paths by Metadata.
- Renamed
shutdown
REST endpoint to_shutdown
and improved its description. - Fixed
Switch
class docstrings.
- Fixed
RuntimeError
when shutting down Kytos.
- New blueprint: EP022 - Kytos reports statistics.
- New method
from_dict
to instantiate Interface, UNI, Link and Switch classes from python dictionary. - Log uncaught exceptions to console and/or log files.
- New log message when handling errors at superuser creation.
- Added file to provide support for Dependabot.
- [docs] New documentation for consistency system.
- [tests] Fix PID value to fix errors in unit test execution (fix #1242).
- [tests] Fix pytest-runner error raised by Scrutinizer CI.
- [docs] Fixed warning in code-block section in auth documentation.
- Updated dependencies.
No changes since rc1.
- Added event to notify when a NApp was loaded.
- [docs][ui] Added
k-notification
component and its event docs. - [docs][ui] Added table that lists the Kytos standard colors.
- [docs] Fixed the cells' order when the Blueprints table is generated.
- [docs] Updated
k-context-panel
andk-table
images and usage examples.
- Added configuration field to change token expiration time in REST API authentication.
- [ui] New UI component: Notification.
- [ui] Added info-panel toggle button in tabs component.
- [ui] Added close button to info-panel component.
- [docs] Added a new "Blueprints" section to the Dev Guide.
- [docs] New section about implementation of compressed and expanded formats for toolbar components UI.
- Refactor method
get_interface_by_port_no
to work with both``v0x01`` andv0x04
port
parameters. - [ui][docs] Updated components' docs: accordion, tooltip and title.
- [ui][docs] Updated usage example for the
event
component - [docs] Updated admin guide with parameter to create a superuser.
- Changed stability badge in PyPI from experimental to beta.
- Fixed double loading of NApps when installing via
kytos napps install
- Fixed
daemon
configuration that was being ignored - [ui] Fixed overlay between tabs component and other components.
- Added authentication to REST methods based on configuration option
- Create
config
field onInterface
- Added new exception
KytosLinkCreationError
- [docs] Created a template blueprint - EP000
- [docs] Added
of_lldp
's new REST Endpoints - [docs] Added "Kytos UI Components" section to Dev Guide
- [docs] New note about the creation of UI folders
- [tests] Added
pydocstyle
as a required linter
- [docs] Updated old blueprints to include standard headers
- [docs] Moved section "Creating a NApp with UI" to the Web-UI documentation
- [docs] Use friendlier
apt
command instead ofapt-get
- [docs] Updated Authentication documentation
- [docs] Updated tutorial "How protect a REST endpoint"
- [tests] Changed tests to use multiple-letter keys in mock link metadata
- Removed hard-coded python3.6 references
- [packaging] Remove the use of distutils from
setup.py
- Improved support for newer versions of Python
- Fixed exception when
kytosd
cannot update the web UI from GitHub - Fixed parsing of
vlan_pool
configuration option - [tests] Fixed test_logs for Python 3.8
- [tests] Fixed automated packaging tests under GitHub Actions
- Added Blueprints section to the "How to Contribute" guide.
- Fixed bug when two NApps had methods with the same name
decorated with the
@rest
decorator. - Fixed authentication URLs in documentation.
- Fixed interface tests.
- Changed
dev
requirements to installrun
requirements. - Changed Makefile to use
python3
instead ofpython3.6
. - Updated
.travis.yml
to use newest pip dependency resolver for tests. - Changed
setup.py
to alert when a test fails on Travis.
- Improve unit tests coverage from 55% to 93%.
- Added new method to handle HTTPException - now it returns a JSON with an error code.
- Added tags decorator to run tests by type and size.
- Added instruction for opening issues with traffic files in Dev Guide.
- Added Pull Request Guidelines to the Developer Guide.
- Fixed duplicated endpoint error in available_vlans method.
- Fixed error when creating an EVC without a Tag.
- Fixed packaging error by changing the
six
version.
- Updated setup.py to use native setuptools install.
- Make speed property checks compliant with OF1.3 spec.
- Updated controller mock method to accept loop parameter.
- Changed API server status HTTP code to 200.
- Updated documentation images, dates and links.
- Added doc listing all the REST APIs available on Kytos Core + NApps
- Fixed random error on concurrent tests, waiting for threads to finish before testing.
- Return the original HTTP error code when a NApp is not found in the NApp server
Link.get_next_available_tag()
now raises an exception (instead of returningFalse
) when there is no available tag
- Added a new
kytos.lib.helpers
module to be used by NApps as an utility for tests. - [kytos/topology] Added persistence for switches and interfaces administrative status (enabled/disabled).
- [kytos/topology] Added REST APIs to enable/disable all interfaces from a switch.
- [kytos/topology] Added listeners for events from the Maintenance NApp.
- [kytos/of_core] Added tag decorators for small/medium/large tests.
- [packaging] Changed Makefile to clean old web-ui builds.
- [kytos/topology] Avoid using flapping links: now a link is considered up only after a specific amount of time (default: 10 seconds).
- [kytos/topology] Fixed switches coordinates on the map.
- Fixed 22 linter issues raised after the pylint upgrade.
- Added shorter README file to use on PyPI description.
- Upgraded versions for all dependencies
- kytosd now create configuration only in post-install - #1042
- Fixed SandboxViolation when installing Kytos as a dependency from PyPI - #494
- Fixed install from wheel package format- #922
- Fixed "There is no config file." error when starting kytosd - #951
- New unit tests for NApps:
- kytos/kronos, coverage increased from 0% to 31%
- kytos/mef_eline, coverage increased from 67% to 70%
- kytos/of_core, coverage increased from 28% to 47%
- New blueprint: EP018 - Kytos testing pipeline and definitions.
- Added long description field for display in pypi.org.
- Fixed Scrutinizer coverage error.
- Increased token expiration time in auth module.
- New etcd backend for the Storehouse NApp (experimental)
- NApps Server now has e-mail verification and password reset for devs
- Added python-openflow unit test coverage section to Kytos Dev guide
- Fixed duplicated logs (#993)
- Fixed exception handling during NApp setup which could cause locks on kytosd shutdown (#1000)
- New Authentication module - REST endpoints can now be protected using the @authenticated decorator.
- New unitests to the Authentication module.
- New /metadata REST endpoint to access package metadata. kytos-utils now uses this to look for version mismatches.
- Blueprint EP018 - Updated endpoints to configure Authentication module.
- Fix kytos installation without virtual env (eg.: sudo).
- New blueprint: EP018 - API Authentication.
- New blueprint: EP019 - Improvements on Statistics Metrics Collections.
- New blueprint: EP020 - Data and Settings Persistence.
- Changed loggers to begin the hierarchy with "kytos."
- Modify the kytos developer mode to check the installation of configuration files.
- Blueprint EP016: Changed layout and improvement ideas.
- Blueprint EP017: More details on OpenFlow errors.
- Interface objects have a new boolean lldp attribute (default True). Other applications can look at this attribute to determine the LLDP behavior.
- Improved installation of dependencies - pinned versions for dependencies in the production and developer install modes.
- This is the stable "fafa" version, based on the last beta pre-releases. No changes since the last rc1.
- Added Makefile for packaging and uploading to PyPI
- Added string representations to Switch and Interface
- New unit test for TCP server exceptions
- pytest is now the default tool for Kytos' unit tests
- Invalid command-line parameters emit warnings instead of halting kytosd start
- Fixed traceback when a switch loses connectivity
- Added REST API endpoints to manage NApps from remote applications
- New kytos/kronos NApp was released. This NApp will be responsible for handling time series data, with initial support for InfluxDB (EXPERIMENTAL). For now on, visit kytos/kronos changelog for updates.
- kytos-utils is now decoupled from kytos core
- Changed default Openflow TCP port to 6653
- Removed diraol's watchdog fork dependency
- Fixed kytos install from PyPI. Now dependencies are properly installed
- Fixed some grammar errors in documentation
- Fixed some linter issues
- Changed some dependencies versions in order to fix security bugs
- Added MEF E-Line Link Up/Down definition blueprint
- Added documentation about using tox for unit tests
- Fixed bug when starting kytosd in background (#893)
- Fixed method get_next_available_tag under concurrent scenarios
- Fixed warning when compiling documentation
- Added vlan_pool configuration on kytos.conf to support mef_eline. Now you can configure available vlans per interface
- Added documentation to describe how to create a Meta Napp
- Added documentation about Unit Tests
- Updated documentation to install python-openflow, kytos-utils and kytos in that order
- Updated documentation to use pip3 instead pip
- Link id is now based on endpoints hashes, instead of a random uuid. This fixes #875
- Removed circular dependency of kytos-utils
- Removed unnecessary comparison on interfaces if they are on the same switch
- Fixed type declaration that broke sphinx-build
- Fixed some linter issues
- Fixed NApps settings reload. Now when you change a NApp settings the reload it will work
- Updated pyyaml and requests requirements versions, in order to fix vulnerabilities
- This is the stable "ernesto" version, based on the last beta pre-releases. No changes since the last rc1.
- Support for meta-napps (EXPERIMENTAL)
- Added support to reuse VLAN pool configurations on Interface
- Added support for serialization of Link instances
- Improved test coverage
- Blueprint EP015 (system tests) improved
- Improved test coverage
- Removed warnings for invalid port speed (fix #754)
- Fixed port speed on web user interface
- Update console to support IPython 7
- Added methods to list all NApp listeners.
- Blueprint EP12.rst updated in order to describe patch and delete operations.
- Fixed compatibility of Python 3.7
- Fixed some linter issues.
- Fixed napps pre-installed with default value.
- Added reload/<username>/<napp_name> endpoint to reload the NApp code
- Added reload/all endpoint to update the NApp code of all NApps
- Kytos console display the kytos version.
- Added method __repr__ on Napp class.
- Added method __eq__ on UNI class.
- UNI and TAG has method as_dict and as_json.
- Added method get_metadata as_dict.
- Added method to return all available vlans.
- Added method to return a specific interface by id.
- Added pre-install napps method.
- Added a better introduction of dev and admin guides.
- Better handling of active/enabled in Switch/Interface/Links entities.
- Better handling of broken napps.
- Refactored load_napps method.
- Refactored get_time to return a datetime with UTC
- Migrated event handler threads to the main asyncio loop.
- Improve documentation to use kytos sphinx theme.
- Some documentation docstrings.
- Added str and repr methods for KytosEvent and Connection classes to be easy to see logging and debugging information.
- Added web/update/<version>/ endpoint to update Kytos Web Interface with a specific version.
- Added asyncio support in tcp server and controller. API Server, ipython, event handlers and event notifications are still running on separate threads.
- Changed the components name provided by Kytos NApps to use the pattern: {username}-{nappname}-{component-section}-{filename}
- Fixed some docstrings and comments
- Added some new blueprints (EP012, EP013 and EP014)
- Now, we have few Entities inside the core (Switch, Interface and Link)
- Each Entity has metadata attribute (a dict)
- Added link attribute to the Interface class
- GenericEntity itself was added in this version also
- Added 'active' and 'enable' flags to GenericEntity (EP013)
- Added 'enable'/'disable' methods to child GenericEntity classes (EP013).
- Define available_tags according to link's interfaces.
- Endpoint ('/ui/all') to display a json with all napps ui components.
- Endpoint ('/ui/<path:filename>') to get file with a specific napp component.
- Now, kytosd is a python module, to make it easy to run with asyncio on the future;
- This pre-release implements EP013 and EP014 as discussed on our last Kytos Dev Meeting.
- Moved Interface class to interface.py file
- Small refactor of Switch class.
- Some bug fixes
- Web User Interface totally updated, with new design and functionality:
- Visual elements reorganized for better experience.
- Better information about switches and interfaces in the network.
- Extending interface functionalities became easier.
- @rest decorator can also be used before @classmethod or @staticmethod.
- Remove napp endpoints when a napp is disabled.
- TCP Server OpenFlow known ports.
- Config to allow other personalized protocol names on TCP Server.
- NNI and UNI attributes to Interface class.
- Interfaces to Switch json output.
- Statistics information for switch interfaces.
- Allow cross origin resource sharing (CORS).
- Now supports speed information from OF 1.3 switchs.
- Generate Events for reconnected switches.
- Dependency installation/update for devs: pip install -Ur requirements/dev.txt. To use cloned kytos repos as dependencies, reinstall those repos with pip install -e . in the end.
- Event name for a new switch. From kytos/core.switches.new to kytos/core.switch.new.
- Flow class from flow module. It was moved to kytos/of_core NApp.
- Some bug fixes in tests.
- Several documentation fixes.
- Several bug fixes.
- Rest API prefix changed to "api/<username>/<nappname>".
- Now displays bandwidth values as bytes.
- Remove rest api endpoint when a NApp is disabled.
- Correctly update interface state and manage interfaces for switches.
- Some bug fixes.
@rest
decorator for REST API methods. Examples:@rest('flow/<flow_id>')
(onlyGET
HTTP method by default);@rest('flows/', methods=['GET', 'POST'])
.
- Guide for developers in documentation.
- Whole documentation updated.
- API URLs renamed:
- For NApps, the pattern is
/api/<username>/<napp>/
+ what is defined in@rest
decorator; - Core endpoints starts with
/api/kytos/core/
. E.g./kytos/config
changed to/api/kytos/core/config
.
- For NApps, the pattern is
- Improved load/unload of NApps.
- Requirements files updated and restructured.
- Yala substitutes Pylama as the main linter checker.
- Method
register_rest_endpoint
ofController
andAPIServer
in favor of@rest
decorator.
- Some bug fixes in tests.
- Several documentation fixes.
- Several bug fixes.
- NAppDirListener to manage (load/unload) NApps when they are enabled or disabled using kytos-utils.
- Improved connection management.
- Documentation updated and improved.
- Improved setup process.
- Some bug fixes.
- Endpoint to display kytos configuration ('/kytos/config/').
- Setting to setup Kytos API Port on kytos.conf ('api_port' default to 8181).
- Documentation Blueprints tree.
- OpenFlow specific code moved to NApps: Kytos now acts as an all-purpose controller.
- Log manager refactored
- Improvements in the web interface style, layout and usability
- Setup process now requires pip
- Kytos documentation now shows a dropdown with each release documentation.
- Web interface: - Fixed memory and CPU usage
- Now Kytos accepts to register different methods [POST, GET, etc] on the same endpoint.
- Now it's possible to start kytos in debug mode using kytosd -D.
- Removed documentation warnings.
- Several bug fixes
- Python bdist_wheel generation to make the install process via 'pip' easier and faster.
- Lockfile (PID-file) creation to prevent multiple instances running at the same time.
- Controller.restart method.
- kytos/tryfirst docker image was created and added to dockerhub.
- An improved console was added to execute python code when the controller is run in foreground.
- Continuous Integration with Code Quality Score and test coverage. (for the Python files in the project).
- Administration User Interface was moved to kytos, and it's accessible at port 8181 when kytos is running.
- Blueprints were moved to kytos/docs/blueprints folder.
- Updated requirements.txt.
- Improvements in TCP Server:
- Now makes sure the switch is fully connected before accepting data.
- Makes sure the switch is still connected before sending any data.
- Uses sendall() to make sure data is being correctly sent.
- NApps module was refactored.
- Improved 'clean' option of setup.py.
- Improved tests and style checks for developers.
- Kytos setup process improved, reading necessary metadata before installing.
- Kytos core package was refactored.
- Documentation updates.
- NApp information is now obtained from kytos.json when loading a NApp.
- Improved log management.
- 'author' attribute, in the NApps context, was replaced by 'username' and will be removed in future releases.
- Friendly messages are now displayed when some exceptions are raised.
- Kytos configuration is now loaded properly from kytos.conf
- Several adjustments and bug fixes.
- Data gathering from switches (i.e. interface speed)
- REST endpoints (i.e. REST API status)
- Sphinxs documentation
- Controller stop/start improvement
- Improved Controller's Rest API (using Flask)
- Connections, interfaces and switches management improvement
- Websocket to send logs to web interface
- Improved log management
- Corrections on setup and installation controller's code
- Improved NApps management - uninstall, disable and unload
- Improved controller's install and setup
- Bootstrapped initial architechture
- Kytos Events managing buffers and handlers
- NApp handling (load/unload/start/shutdown)
- TCPServer and TCPHandler
- Added basic config class