Matters Of Ethics, Trust, And Potential Liability For Autonomous Systems

Abstract

The objective of this panel was to discuss issues related to the development and use of autonomous systems, with specific focus on the overriding themes of ethical considerations and potential liability for Human Factors and Ergonomics (HF/E) professionals who are involved in their development. Chris Brill provided opening remarks to frame the discussion and introduce the panelists. James Bliss discussed legal implications related to our collective penchant for developing conservative, false-alarm prone automation. Peter Hancock advocated for human-centered constraints on autonomous systems, as they may, one day, pose an existential threat to humanity. Dietrich Manzey discussed ethical considerations for autonomous systems, including how design can encourage ethical user behavior. Joachim Meyer argued that HF/E professionals have an obligation to help designers understand the ethical implications of poor design, particularly in the context of autonomous systems. Lastly, Alison Vredenburgh provided thoughts on potential liability for HF/E professionals, particularly in light of the relative newness of autonomous systems. The panel then turned to facilitated discussion with panelists and audience members. Specific themes included the boundaries of our responsibilities as HF/E professionals for ill-conceived or morally-objectionable systems, potential implications of manipulating user trust through design, cross-cultural perspectives on public acceptance and legal peril, and how concerns might differ by domain (e.g., medical vs.combat vs. manufacturing). The session concluded with panelists summarizing how ethics influence design and recommendations for how HF/E professionals can potentially protect themselves from legal liability for mishaps involving autonomous systems they helped develop.

Publication Date

1-1-2016

Publication Title

Proceedings of the Human Factors and Ergonomics Society

Number of Pages

308-312

Document Type

Article; Proceedings Paper

Personal Identifier

scopus

DOI Link

https://doi.org/10.1177/1541931213601070

Socpus ID

85021816631 (Scopus)

Source API URL

https://api.elsevier.com/content/abstract/scopus_id/85021816631

This document is currently not available here.

Share

COinS