Scope
Not just verification, evaluation, or software engineering, but a holistic view of how we engineer reliable autonomous systems. For the purposes of this conference, “autonomous systems” is synonymous with "systems that make their own decisions and can take their own actions," and includes everything from autonomous decision-making software running in a virtual environment to teams of autonomous robots operating in a distributed physical environment, everything from simple reactive systems with no planning capability to complex systems incorporating machine learning and reasoning engines. Contributions addressing aspects of reliability across the entire life cycle of the system are welcome.
Relevant Topics
General Understanding of Autonomy
- Understanding the issues of autonomy, especially under the excess uncertainty of complex deployments (uncertainty and complexity permeate autonomous system design and operation - reliability requires that we address them)
- Development of scientific foundations for autonomy that can drive the development of reliable systems
- Application of the scientific method to autonomous systems experimentation and evaluation
- Reproducibility, replicability, and generalizability of autonomous systems experiments
- Research challenges and roadmaps for future development
Specification and User Needs
- Defining, ensuring, and assessing system properties (safety, security, functionality, reliability, dependability, trustworthiness, …)
- Stakeholder communication, expressing user needs and experiences, designer decisions and assumptions, test and evaluation results, operational expectations
- Specification of requirements
- System purpose, goals, and expectations as expressed by designers, testers, certification agents, users, customers, bystanders, etc.
- Establishing trust and understanding of autonomous system behavior
Design and Systems Engineering for Reliability
- System engineering and design principles (including reliable / dependable robot control architectures, systems that incorporate a range of artificial intelligence capabilities, and assurance for design as well as design for assurance)
- Fault handling (including prediction, detection, isolation, identification, response, recovery, prevention, tolerance, removal) and runtime methods for recognition and recovery (monitoring, diagnosis, reonfigurability, verification, assurance) - runtime systems that provide resilience and dynamic functionality - autonomy that exists within the system to handle cases that fall outside nominal bounds)
- Understanding and analysing trade-offs in system development (e.g. efficiency vs. transparency/verifiability/explainability; design-time vs. run-time; "Hoping for the best" vs. "Expecting the worst")
Assessing and Communicating Reliability
- Test and evaluation techniques, principles, methods, tools, etc.
- Verification and validation of autonomy (including techniques, processes, principles, methods, tools, verification of decision making, handling of unknown unknowns, uncertainty, and complexity)
- Verification and validation of test and evaluation techniques, tools, etc.
- Assurance and evidence (including types of evidence, confidence, evidence gathering, specific assurance cases for autonomous systems, structuring and design principles for assurance case development / integration of evidence into assurance)
- Measurement and metrics
- Mapping from evaluation results to operational performance
- Context and impact of verification and evaluation on reliability
- Runtime evaluation and assessment
- Perceived and actual reliability
Reliability in Context
- Licensure, regulatory approval, and certification
- Standards and industry benchmarks
- Legal and ethical considerations (including insurance and liability as well as government and public service considerations)
- Industrial case studies and considerations
- Life cycle considerations and methodologies (impact of reliability concerns on need identification, specification, design, development, evaluation and assessment of trustworthiness, operation, decommission)
- Case studies (focused in specific application domains (e.g. healthcare diagnosis and intervention / autonomous driving / domestic robotics), specific environmental domains (e.g. space / maritime / volcanos / office buildings) and specific research domains (e.g. controls / perception))
- Problem sets, reliability benchmarks and competitions
