Desarrollo de un algoritmo de navegación autónoma para uavs basado en objetivos dados usando técnicas de aprendizaje por refuerzo profundo

Calderon Chavez, Juan Manuel; Guarnizo Marín, José Guillermo

doi:https://doi.org/10.15332/dt.inv.2020.01522

Desarrollo de un algoritmo de navegación autónoma para uavs basado en objetivos dados usando técnicas de aprendizaje por refuerzo profundo

dc.contributor.author	Calderon Chavez, Juan Manuel
dc.contributor.author	Guarnizo Marín, José Guillermo
dc.contributor.cvlac	http://scienti.colciencias.gov.co:8081/cvlac/visualizad or/generarCurriculoCv.do?cod_rh=0000380938
dc.contributor.cvlac	http://scienti.colciencias.gov.co:8081/cvlac/visualizador/generarCurriculoCv.do?cod_rh=0000855847
dc.contributor.googlescholar	https://scholar.google.com/citations?user=095RddUAAAAJ&hl=en
dc.contributor.googlescholar	https://scholar.google.com/citations?hl=es&user=_mObTPkAAAAJ
dc.contributor.orcid	https://orcid.org/0000-0002-4471-3980
dc.contributor.orcid	https://orcid.org/0000-0002-8401-4949
dc.date.accessioned	2020-04-20T17:15:19Z
dc.date.available	2020-04-20T17:15:19Z
dc.date.issued	2019-08
dc.description	Los desastres naturales y las guerras son algunos de los peores eventos que la humanidad ha tenido que enfrentar, ya que en este tipo de situaciones es casi imposible evacuar a las personas en el área afectada, causando muchas más muertes y un impacto devastador. Por lo anterior se hace necesario el uso de robots autónomos que colaboren en la búsqueda y rescate de víctimas humanas en zonas de desastre. Este proyecto de investigación propone el uso de técnicas de aprendizaje por refuerzo profundo o "Deep Reinforcement Lerning - DRL" para proporcionar habilidades de navegación autónoma y adaptación en ambientes desconocidos a un robot aéreo no tripulado. Se propone el uso de información visual como sistema de sensado del ambiente. Dado que el ambiente es no estructurado y desconocido para el agente robótico no se construirán mapas ni se intentará seguir un mapa estrictamente. La navegación se basa en el seguimiento y evasión de objetivos dados, entre los cuales se cuentan personas, arboles, fuego y demás objetos que sean considerablemente identificables mediante información visual en una zona de desastre.	spa
dc.description.abstract	Natural disasters and wars are some of the worst events that the humanity has had to face, since in this type of situation it is almost impossible evacuate people in the affected area, causing many more deaths and an impact devastating. Therefore, it is necessary to use autonomous robots that collaborate in the search and rescue of human victims in disaster areas. This project of research proposes the use of deep reinforcement learning techniques or "Deep Reinforcement Lerning - DRL "to provide navigation skills autonomous and adaptation in unknown environments to an unmanned aerial robot. I know proposes the use of visual information as a sensing system for the environment. Given the the environment is unstructured and unknown to the robotic agent will not be built maps nor will any attempt be made to strictly follow a map. Navigation is based on the follow-up and evasion of given objectives, which include people, trees, fire and other objects that are considerably identifiable by visual information in a disaster area.	spa
dc.description.domain	http://unidadinvestigacion.usta.edu.co	spa
dc.format.mimetype	application/pdf
dc.identifier.doi	https://doi.org/10.15332/dt.inv.2020.01522	spa
dc.identifier.uri	http://hdl.handle.net/11634/22649
dc.publisher.branch	CRAI-USTA Bogotá	spa
dc.relation.references	Casper, Jennifer, and Robin R. Murphy. "Human-robot interactions during the robot-assisted urban search and rescue response at the world trade center." IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 33, no. 3 (2003): 367-385.	spa
dc.relation.references	Kitano, Hiroaki, Satoshi Tadokoro, Itsuki Noda, Hitoshi Matsubara, Tomoichi Takahashi, Atsuhi Shinjou, and Susumu Shimada. "Robocup rescue: Search and rescue in large-scale disasters as a domain for autonomous agents research."	spa
dc.relation.references	Atkeson, Christopher G., Benzun P. Wisely Babu, Nandan Banerjee, Dmitry Berenson, Christoper P. Bove, Xiongyi Cui, Mathew DeDonato et al. "No falls, no resets: Reliable humanoid behavior in the DARPA robotics challenge." In Humanoid Robots (Humanoids), 2015 IEEE-RAS 15th International Conference on, pp. 623-630. IEEE, 2015	spa
dc.relation.references	Christensen, Henrik I., T. Batzinger, K. Bekris, K. Bohringer, J. Bordogna, G. Bradski, O. Brock, J. Burnstein, T. Fuhlbrigge, and R. Eastman. "A roadmap for us robotics: from internet to robotics." Computing Community Consortium (2009).	spa
dc.relation.references	Christensen, H. "A Roadmap for US Robotics from Internet to Robotics, 2016 edn." Sponsored by National Science Foundation & University of California, San Diego (2016).	spa
dc.relation.references	Robotics, S. P. A. R. C. "Robotics 2020 multi-annual roadmap for robotics in Europe." SPARC Robotics, EU-Robotics AISBL, The Hauge, The Netherlands, accessed Feb 5 (2016): 2018.	spa
dc.relation.references	Gerald Tesauro. Temporal difference learning and td-gammon. Communications of the ACM, 38(3):58–68, 1995	spa
dc.relation.references	Quesada, Wilson, “Generación de comportamientos de enjambre en robots móviles a través del uso del aprendizaje por refuerzo.”, p. 81, 2018	spa
dc.relation.references	Mirowski, P.W., Pascanu, R., Viola, F., Soyer, H., Ballard, A.J., Banino, A., Denil, M., Goroshin, R., Sifre, L., Kavukcuoglu, K., Kumaran, D., & Hadsell, R. “Learning to Navigate in Complex Environments.” CoRR (2017), abs/1611.03673.	spa
dc.relation.references	Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M.A. “Playing Atari with Deep Reinforcement Learning.” CoRR (2013), abs/1312.5602	spa
dc.relation.references	Arulkumaran, Kai, et al. "A brief survey of deep reinforcement learning." arXiv preprint arXiv:1708.05866 (2017).	spa
dc.relation.references	J. Borenstein and Y. Koren, “Real-time obstacle avoidance for fast mobile robots,” IEEE Trans. on Systems, Man and Cybernetics, 1989	spa
dc.relation.references	F. Dayoub, T. Morris, B. Upcroft, and P. Corke, “Vision-only autonomous navigation using topometric maps,” in IROS, 2013.	spa
dc.relation.references	D. Wooden, “A guide to vision-based map building,” IEEE Robotics and Automation Magazine, 2006	spa
dc.relation.references	A. De, K. S. Bayer, and D. E. Koditschek, “Active sensing for dynamic, non-holonomic, robust visual servoing,” in ICRA, 2014.	spa
dc.relation.references	P. Saeedi, P. D. Lawrence, and D. G. Lowe, “Vision-based 3-d trajectory tracking for unknown environments,” T-RO, 2006	spa
dc.relation.references	K. Konolige, J. Bowman, J. Chen, P. Mihelich, M. Calonder, V. Lepetit, and P. Fua, “View-based maps,” Intl. J. of Robotics Research, 2010.	spa
dc.relation.references	S. Phillips, A. Jaegle, and K. Daniilidis, “Fast, robust, continuous monocular egomotion computation,” in ICRA, 2016	spa
dc.relation.references	N. Kohl and P. Stone, “Policy gradient reinforcement learning for fast quadrupedal locomotion,” in ICRA, 2004.	spa
dc.relation.references	J. Peters and S. Schaal, “Reinforcement learning of motor skills with policy gradients,” Neural Networks, 2008.	spa
dc.relation.references	J. Michels, A. Saxena, and A. Ng, “High speed obstacle avoidance using monocular vision and reinforcement learning,” in ICML, 2005	spa
dc.relation.references	T. Kollar and N. Roy, “Trajectory optimization using reinforcement learning for map exploration,” Intl. J. of Robotics Research, 2008	spa
dc.relation.references	V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al., “Humanlevel control through deep reinforcement learning,” Nature, 2015	spa
dc.relation.references	Botvinick, Mathew, et al. "Reinforcement learning, fast and slow." Trends in cognitive sciences (2019)	spa
dc.relation.references	D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, et al., “Mastering the game of go with deep neural networks and tree search,” Nature, 2016	spa
dc.relation.references	Barratt, Jeffrey, and Chuanbo Pan. "Playing Go without Game Tree Search Using Convolutional Neural Networks." arXiv preprint arXiv:1907.04658 (2019)	spa
dc.relation.references	Barriga, Nicolas A., et al. "Improving RTS Game AI by Supervised Policy Learning, Tactical Search, and Deep Reinforcement Learning." IEEE Computational Intelligence Magazine 14.3 (2019)	spa
dc.relation.references	Lan, Li-Cheng, et al. "Multiple Policy Value Monte Carlo Tree Search." arXiv preprint arXiv:1905.13521 (2019)	spa
dc.relation.references	V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. P. Lillicrap, T. Harley, D. Silver, and K. Kavukcuoglu, “Asynchronous methods for deep reinforcement learning,” in ICML, 2016.	spa
dc.relation.references	Gong, Xudong, et al. "Parallelized Synchronous Multi-agent Deep Reinforcement Learning with Experience Replay Memory." 2019 IEEE International Conference on Service-Oriented System Engineering (SOSE). IEEE, 2019.	spa
dc.relation.references	Weitkamp, Laurens, Elise van der Pol, and Zeynep Akata. "Visual Rationalizations in Deep Reinforcement Learning for Atari Games." arXiv preprint arXiv:1902.00566 (2019).	spa
dc.relation.references	Zhao, Xingyu, et al. "Applications of asynchronous deep reinforcement learning based on dynamic updating weights." Applied Intelligence 49.2 (2019)	spa
dc.relation.references	S. Levine, C. Finn, T. Darrell, and P. Abbeel, “End-to-end training of deep visuomotor policies,” JMLR, 2016.	spa
dc.relation.references	Abolghasemi, Pooya, et al. "Pay Attention!-Robustifying a Deep Visuomotor Policy Through Task-Focused Visual Attention." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019.	spa
dc.relation.references	Kim, Seongun, et al. "Implementation of End-to-End Training of Deep Visuomotor Policies for Manipulation of a Robotic Arm of Baxter Research Robot." Journal of Korea Robotics Society14.1. 2019	spa
dc.relation.references	Luo, Wenhan, et al. "End-to-end active object tracking and its real-world deployment via reinforcement learning." IEEE transactions on pattern analysis and machine intelligence, 2019	spa
dc.relation.references	A. A. Rusu, S. G. Colmenarejo, C¸. Gulc¸ehre, G. Desjardins, J. Kirk-¨ patrick, R. Pascanu, V. Mnih, K. Kavukcuoglu, and R. Hadsell, “Policy distillation,” in ICLR, 2016.	spa
dc.relation.references	E. Parisotto, L. J. Ba, and R. Salakhutdinov, “Actor-mimic: Deep multitask and transfer reinforcement learning,” in ICLR, 2016	spa
dc.relation.references	Sutton, Richard S., and Andrew G. Barto. Reinforcement learning: An introduction. MIT press, 2018.	spa
dc.relation.references	Fenjiro, Youssef, and Houda Benbrahim. "Deep reinforcement learning overview of the state of the art." Journal of Automation Mobile Robotics and Intelligent Systems 12 (2018).	spa
dc.relation.references	Li, Yuxi. "Deep reinforcement learning: An overview." arXiv preprint arXiv:1701.07274, 2017.	spa
dc.rights	Atribución-NoComercial-SinDerivadas 2.5 Colombia
dc.rights.uri	http://creativecommons.org/licenses/by-nc-nd/2.5/co/
dc.subject.keyword	Autonomous robots	spa
dc.subject.keyword	Unmanned aerial robots	spa
dc.subject.keyword	Deep reinforcement learning	spa
dc.subject.proposal	Robots autónomos	spa
dc.subject.proposal	Robots aéreos no tripulados	spa
dc.subject.proposal	Aprendizaje por refuerzo profundo	spa
dc.title	Desarrollo de un algoritmo de navegación autónoma para uavs basado en objetivos dados usando técnicas de aprendizaje por refuerzo profundo	spa
dc.type.category	Formación de Recurso Humano para la Ctel: Proyecto ejecutado con investigadores en empresas, industrias y Estado	spa