Git version: 2.42.0.windows.2
Error: ssh: connect to host github.com port 22: Connection refused
I have been able to update and publish website content normally before, but today when I use hexo d to update my blog, I encountered an error as shown below:
Deep Q-Network (DQN) is a deep reinforcement learning method, first proposed by Google DeepMind team in 2013 [Paper]. This algorithm introduces a neural network to replace the cumulative expected return storage form in the Q-learning algorithm, and can effectively solve problems with high-dimensional state spaces. The DeepMind team creatively combines convolutional neural networks in deep learning and Q-learning in reinforcement learning to train agents to learn how to control characters in Atari games to achieve good game performance.
The DQN algorithm takes the state of the agent as the input of the neural network, and then uses the neural network to calculate the Q values corresponding to all actions.
This project is also part of my master's thesis, and it includes two aspects: global path planning of each robot and collaborative planning of multiple robots. My master's thesis (Written in Chinese) can be downloaded here.
Abstract:
To solve the routing problem in Mobile Robot Fulfillment System, firstly, a method based on traditional A* algorithm is proposed, and then a method based on improved A* algorithm considering task priority and utilization frequency of path nodes is further designed. Finally, a Markov decision process describing the problem is established, and a method based on Q-learning is proposed. Results of computational experiments show that these algorithms can quickly solve the problem. The methods based on the traditional A* algorithm and Q-learning can both obtain the shortest paths, and the method based on improved A* algorithm can effectively balance the traffic flow and significantly reduce the number of potential collisions.
More details about this project can be found in this paper. Here is a brief introduction to the project. This project is part of my master's thesis. My master's thesis (Written in Chinese) can be downloaded here.
Abstract:
This paper studies a task scheduling problem in the context of the mobile robot fulfillment system (MRFS), a parts-to-picker storage system where mobile robots bring movable racks to workstations. It determines the assignment of tasks of transporting racks to a fleet of robots with the objective of makespan minimization. A mixed integer programming model is presented to describe the problem. Aimed at quickly finding good solutions to this NP-hard problem, two heuristic rules and an ant colony optimization algorithm are developed. Computational experiments are conducted to evaluate the performance of the proposed heuristic solution procedures. It shows that the ant colony optimization algorithm generally has the best performance.
A mixed integer programming model is presented to describe the problem as follows:
This was a course project during my undergraduate studies and was published in a Chinese journal. The course paper can be downloaded here. The following is a translation of the paper.
Abstract:
The queuing problem in hospital is becoming more and more serious. This paper investigates the queuing system of registration window of a hospital in Nanjing. The data of arrival time interval and service time of each window are collected, and its distributions are fitted by Matalab. The model of the system is set up and simulated with FlexSim. According to the simulation results, the optimization scheme is put forward. By increasing the number of windows opened to patients from 3 to 4, the average waiting time is reduced by 430 seconds, and the queuing time is reduced by 69%.
Key words: hospital；registration window；queueing system；simulation；FlexSim
MongoDB is a NoSQL database product. MongoDB uses collections to organize documents, and each document is composed of key-value pairs.
More information about MongoDB: https://en.wikipedia.org/wiki/MongoDB
Download and install Mongo from here: Download MongoDB Community Server | MongoDB
MQTT (Message Queuing Telemetry Transport), is a lightweight publish/subscribe messaging protocol.
More information about MQTT: https://en.wikipedia.org/wiki/MQTT
Install paho-mqtt library in Python:
1 | pip install paho-mqtt |
I used PyQt5 in Python to develop a GUI for a simple program. After the development was completed, I packaged the program to generate an exe file. But when I run this exe file and click the button on the GUI, a new window will pop up. If I click the button repeatedly, new windows keep being generated. This problem does not occur when debugging and running the program in PyCharm.
Install the Pyinstaller package via pip with the following command:
1 | pip install pyinstaller |
More information about PyInstaller: https://pyinstaller.org/en/stable/
