Hogwarts

zxy's personal website

0%

Setup python and prepare docker images on linux

Recently, I am preparing the docker image which is necessary for one course project. The OS version is Linux CentOS. The docker image should contain all necessary libraries that the project needs.

Step1. Install Python3.7

The following links tell users how to install python3 on linux

https://blog.csdn.net/qq_35304570/article/details/80302872

https://www.cnblogs.com/kimyeee/p/7250560.html

I use python 3.7.8 here

1
wget https://www.python.org/ftp/python/3.7.8/Python-3.7.8.tgz
Encountered error
1
zipimport.ZipImportError: can’t decompress data

This error is caused by lack of necessary dependencies. It is important to install all dependencies in the first step.

https://blog.csdn.net/u014749862/article/details/54430022

https://www.cnblogs.com/jimc/p/10218062.html

Step2. Prepare docker images

https://msd.misuland.com/pd/3065794831805579512

Encountered error

When stating the docker using

1
2
3
sudo systemctl start docker

A dependency job for docker.service failed. See 'journalctl -xe' for details.

a brain f**king error popped out

I hav tried several methods that I can found, spend about 40 min, still in vain.

Reinstall my docker

Then I decide to reinstall it according to this link to see whether it works or not.

https://www.cnblogs.com/myzony/p/9071210.html

And then, emm…, another error occurred.

1
2
3
4
  File "/usr/bin/yum", line 30
except KeyboardInterrupt, e:
^
SyntaxError: invalid syntax

This was caused by the python version. /usr/bin/yum is written in python2 snytax, whereas I have upgrade my python to python3 and setup the soft link “python => python3”. Fortunately, the following link solves my problem. And during the process of reinstalling docker, I use this trick to solve several similar errors.

https://stackoverflow.com/questions/11213520/yum-crashed-with-keyboard-interrupt-error

It still didn’t solve my problem after I completely uninstall and install docker for several times. I got the same error when I start the docker. So I decide to investigate on this more carefully, first thing is to check the exact error.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
[root@izwz9h66f1dwk5qi56kfs0z ~]# systemctl status docker.service -l
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
Active: inactive (dead)
Docs: https://docs.docker.com

Feb 28 12:36:09 izwz9h66f1dwk5qi56kfs0z systemd[1]: Dependency failed for Docker Application Container Engine.
Feb 28 12:36:09 izwz9h66f1dwk5qi56kfs0z systemd[1]: Job docker.service/start failed with result 'dependency'.
Feb 28 13:26:58 izwz9h66f1dwk5qi56kfs0z systemd[1]: [/usr/lib/systemd/system/docker.service:37] Unknown lvalue 'TasksMax' in section 'Service'
Feb 28 13:26:58 izwz9h66f1dwk5qi56kfs0z systemd[1]: [/usr/lib/systemd/system/docker.service:37] Unknown lvalue 'TasksMax' in section 'Service'
Feb 28 13:28:12 izwz9h66f1dwk5qi56kfs0z systemd[1]: Dependency failed for Docker Application Container Engine.
Feb 28 13:28:12 izwz9h66f1dwk5qi56kfs0z systemd[1]: Job docker.service/start failed with result 'dependency'.
Feb 28 13:28:54 izwz9h66f1dwk5qi56kfs0z systemd[1]: Dependency failed for Docker Application Container Engine.
Feb 28 13:28:54 izwz9h66f1dwk5qi56kfs0z systemd[1]: Job docker.service/start failed with result 'dependency'.
Feb 28 13:30:44 izwz9h66f1dwk5qi56kfs0z systemd[1]: Dependency failed for Docker Application Container Engine.
Feb 28 13:30:44 izwz9h66f1dwk5qi56kfs0z systemd[1]: Job docker.service/start failed with result 'dependency'.

I start the docker again, and use “journalctl -xe” to see detailed error information.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
[root@izwz9h66f1dwk5qi56kfs0z ~]# systemctl start docker
A dependency job for docker.service failed. See 'journalctl -xe' for details.
[root@izwz9h66f1dwk5qi56kfs0z ~]# journalctl -xe
Feb 28 13:34:35 izwz9h66f1dwk5qi56kfs0z systemd[1]: containerd.service: main process exited, code=exited, status=203/EXEC
Feb 28 13:34:35 izwz9h66f1dwk5qi56kfs0z systemd[1]: Failed to start containerd container runtime.
-- Subject: Unit containerd.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit containerd.service has failed.
--
-- The result is failed.
Feb 28 13:34:35 izwz9h66f1dwk5qi56kfs0z systemd[1]: Dependency failed for Docker Application Container Engine.
-- Subject: Unit docker.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit docker.service has failed.
--
-- The result is dependency.
Feb 28 13:34:35 izwz9h66f1dwk5qi56kfs0z systemd[1]: Job docker.service/start failed with result 'dependency'.
Feb 28 13:34:35 izwz9h66f1dwk5qi56kfs0z systemd[1]: Unit containerd.service entered failed state.
Feb 28 13:34:35 izwz9h66f1dwk5qi56kfs0z systemd[1]: containerd.service failed.
Feb 28 13:34:35 izwz9h66f1dwk5qi56kfs0z polkitd[398]: Unregistered Authentication Agent for unix-process:8386:350732 (system bus name :1.505, object
Feb 28 13:34:40 izwz9h66f1dwk5qi56kfs0z systemd[1]: containerd.service holdoff time over, scheduling restart.
Feb 28 13:34:40 izwz9h66f1dwk5qi56kfs0z systemd[1]: Starting containerd container runtime...
-- Subject: Unit containerd.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit containerd.service has begun starting up.
Feb 28 13:34:40 izwz9h66f1dwk5qi56kfs0z systemd[8413]: Failed at step EXEC spawning /usr/bin/containerd: No such file or directory
-- Subject: Process /usr/bin/containerd could not be executed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- The process /usr/bin/containerd could not be executed and failed.
--
-- The error number returned by this process is 2.
Feb 28 13:34:40 izwz9h66f1dwk5qi56kfs0z systemd[1]: containerd.service: main process exited, code=exited, status=203/EXEC
Feb 28 13:34:40 izwz9h66f1dwk5qi56kfs0z systemd[1]: Failed to start containerd container runtime.
-- Subject: Unit containerd.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit containerd.service has failed.
--
-- The result is failed.
Feb 28 13:34:40 izwz9h66f1dwk5qi56kfs0z systemd[1]: Unit containerd.service entered failed state.
Feb 28 13:34:40 izwz9h66f1dwk5qi56kfs0z systemd[1]: containerd.service failed.

The main error is the following:

  • containerd.service: main process exited, code=exited, status=203/EXEC

  • Failed at step EXEC spawning /usr/bin/containerd: No such file or directory

My understanding is that when the system want to boot docker, it failed to start containerd container at runtime. The above error is the reason. Then the system try to “Starting containerd container runtime…” again, this time the following errors occurred. According to what I have found, it seems that this is not a bug. This problem is due to the incorrect environment or path settings of the user.

Anyway, after trying for 5 hours, I give up fixing this bug. This is quite annoying to work all day on this problem with no progress. I reset all the server and start everything from scratch. If time allowed, in the future I will try to solve this problem. But now I have tons of things to do…

Therefore, the first step should be Step0. Install Docker and then continue step1,2

Now, after the stupid “all start from scratch”. I successfully start my docker. And I am also aware of the lack of knowledge of linux. Previously I add one new user named zxy in centos. However, I need to switch back and forth between root and zxy, which brings me lots of trouble since the authority of different usr differs. Every time when I want to modify some system related files, I have to use sudo and input my password. So this time, I will only focus on root user since there will be nobody except me using this server.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
[root@izwz9h66f1dwk5qi56kfs0z ~]# systemctl start docker
[root@izwz9h66f1dwk5qi56kfs0z ~]# docker version
Client: Docker Engine - Community
Version: 20.10.4
API version: 1.41
Go version: go1.13.15
Git commit: d3cb89e
Built: Thu Feb 25 07:06:20 2021
OS/Arch: linux/amd64
Context: default
Experimental: true

Server: Docker Engine - Community
Engine:
Version: 20.10.4
API version: 1.41 (minimum version 1.12)
Go version: go1.13.15
Git commit: 363e9a8
Built: Thu Feb 25 07:04:45 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.3
GitCommit: 269548fa27e0089a8b8278fc4fc781d7f65a939b
runc:
Version: 1.0.0-rc92
GitCommit: ff819c7e9184c13b7c2607fe6c30ae19403a7aff
docker-init:
Version: 0.19.0
GitCommit: de40ad0
[root@izwz9h66f1dwk5qi56kfs0z ~]#

Build the docker file using the following commands.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# base image
FROM python:3.7

# add the ./recommender into container folder /recommender
# later we can go into the container to see this
ADD ./recommender /recommender
ADD ./requirements.txt /tmp/requirements.txt

# install dependencies
RUN pip install -r /tmp/requirements.txt

# set the working directory to be /recommender
WORKDIR /recommender

# expose the port
EXPOSE 4001

# when starting the container
# use python3 to execute the py shell script of the specified path
CMD ["python3", "/recommender/main.py"]
CMD echo "---end---"
CMD /bin/bash

This may stuck in pip install, so we can simply ignore the dependency installation. We can first create a simple image, create a container using that image, enter it and install (mirror accelerated), then build a new image based on this container.

Up till now, the python environment and docker image construction is done.

Some other problems

https://blog.csdn.net/eleanoryss/article/details/70207767

https://www.cnblogs.com/hongzg1982/articles/2101792.html