This title appears in the Scientific Report :
2021
Please use the identifier:
http://hdl.handle.net/2128/27904 in citations.
Deep Learning for Prediction and Control of Cellular Automata in Unreal Environments
Deep Learning for Prediction and Control of Cellular Automata in Unreal Environments
In this thesis, we show the ability of a deep convolutional neural network to understand the underlying transition rules of twodimensional cellular automata by pure observation. To do so, we evaluate the network on a prediction task, where it has to predict the next state of some cellular automata,...
Saved in:
Personal Name(s):  Aach, Marcel (Corresponding author) 

Contributing Institute: 
Jülich Supercomputing Center; JSC 
Imprint: 
2021

Physical Description: 
76 pages 
Dissertation Note: 
Masterarbeit, University of Cologne, 2021 
Document Type: 
Master Thesis 
Research Program: 
Enabling Computational & DataIntensive Science and Engineering 
Link: 
OpenAccess 
Publikationsportal JuSER 
LEADER  04746nam a2200409 a 4500  

001  892991  
005  20210623133408.0  
024  7  a 2128/27904 2 Handle  
037  a FZJ202102488  
100  1  a Aach, Marcel 0 P:(DEJuel1)180916 b 0 e Corresponding author  
245  a Deep Learning for Prediction and Control of Cellular Automata in Unreal Environments f  20210212  
260  c 2021  
300  a 76 pages  
502  a Masterarbeit, University of Cologne, 2021 c University of Cologne b Masterarbeit d 2021  
520  a In this thesis, we show the ability of a deep convolutional neural network to understand the underlying transition rules of twodimensional cellular automata by pure observation. To do so, we evaluate the network on a prediction task, where it has to predict the next state of some cellular automata, and a control task, where it has to intervene in the evolution of a cellular automaton to achieve a state of standstill. The cellular automata we use in this case are based on the classical Game of Life by John Conway and implemented in the Unreal Engine. With the usage of the Unreal Engine for data generation, a technical pipeline for processing output images with neural networks is established.Cellular automata in general are chaotic dynamical systems, making any sort of prediction or control very challenging, but using convolutional neural networks to exploit the locality of their interactions is a promising approach to solve these problems. The network we present in this thesis follows the EncoderDecoder structure and features residual skip connections that serve as shortcuts in between the different layers. Recent advancements in the field of image recognition and segmentation have shown that both of these aspects are the key to success.The evaluation of the prediction task is split into several levels of generalization: we train the developed network on trajectories of several hundred different cellular automata, varying in their transition rules and neighborhood sizes. Results on a test set show that the network is able to learn the rules of even more complex cellular automata (with an accuracy of ≈ 93%). To some extent, it is even able to interpolate and generalize to completely unseen rules (with an accuracy of ≈ 77%). A qualitative investigation shows that static rules (not forcing many changes in between time steps) are among the easiest to predict.For the control task, we combine the encoder part of the developed neural network with a reinforcement agent and train it to stop all movements on the grid of the cellular automata as quickly as possible. To do so, the agent can change the state of a single cell per time step. A comparison between giving back rewards to agents continuously and giving them only in the case of success or failure shows that Proximal Policy Optimization agents do better with receiving sparse rewards while Deep QNetwork agents fare better with continuously receiving them. Both algorithms beat random agents on training data, but their generalization ability remains limited.  
856  4  u http://juser.fzjuelich.de/record/892991/files/Master_Thesis_MarcelAach.pdf y OpenAccess  
909  C  O  o oai:juser.fzjuelich.de:892991 p openaire p open_access p VDB p driver p dnbdelivery 
910  1  a Forschungszentrum Jülich 0 I:(DE588b)50084628 k FZJ b 0 6 P:(DEJuel1)180916  
913  0  a DEHGF b Key Technologies l Supercomputing & Big Data 1 G:(DEHGF)POF3510 0 G:(DEHGF)POF3511 3 G:(DEHGF)POF3 2 G:(DEHGF)POF3500 4 G:(DEHGF)POF v Computational Science and Mathematical Methods x 0  
913  1  a DEHGF b Key Technologies l Engineering Digital Futures – Supercomputing, Data Management and Information Security for Knowledge and Action 1 G:(DEHGF)POF4510 0 G:(DEHGF)POF4511 3 G:(DEHGF)POF4 2 G:(DEHGF)POF4500 4 G:(DEHGF)POF v Enabling Computational & DataIntensive Science and Engineering x 0  
914  1  y 2021  
915  a OpenAccess 0 StatID:(DEHGF)0510 2 StatID  
980  a master  
980  a VDB  
980  a UNRESTRICTED  
980  a I:(DEJuel1)JSC20090406  
980  1  a FullTexts  
536  a Enabling Computational & DataIntensive Science and Engineering 0 G:(DEHGF)POF4511 c POF4511 x 0 f POF IV  
336  a masterThesis 2 DRIVER  
336  a MASTERSTHESIS 2 BibTeX  
336  a PSP 2.8.2 Magnets 0 2 2 EndNote  
336  a Output Types/Supervised Student Publication 2 DataCite  
336  a SUPERVISED_STUDENT_PUBLICATION 2 ORCID  
336  a Master Thesis b master m master 0 PUB:(DEHGF)19 s 1622611184_10531 2 PUB:(DEHGF)  
920  l yes  
920  k Jülich Supercomputing Center; JSC 0 I:(DEJuel1)JSC20090406 l Jülich Supercomputing Center x 0  
990  a Aach, Marcel 0 P:(DEJuel1)180916 b 0 e Corresponding author 