OMS运维管理平台搭建很多坑的解决方案分享

OMS运维管理平台搭建

一、基础OMS系统部署
1.Django下载地址下载地址:
2.环境:
3.pip 安装
- pip下载
4.Saltstack安装
5.MariaDB安装
6.nginx环境
7.Salt API安装配置
8.Django开发环境搭建
二、部署进阶
三、参考文章

一、基础OMS系统部署

1.Django下载地址下载地址:

https://www.djangoproject.com/download/

因为找不到Django1.11.6，使用pip来下载非常的慢，
pip更新为最新版本

所以这里就直接在官网下载Unsupported previous releases (no longer receive security updates or bug fixes)中的1.11.29

2.环境:

操作系统：redhat7.2

需要的环境：

Django 1.11.29
Python 2.7.5
Saltstack
Salt API
MariaDB
MySQL-python
Nginx
uwsgi

3.pip 安装

pip下载

pip官网：

https://pypi.python.org/pypi/pip

pip下载：

https://pypi.python.org/packages/11/b6/abcb525026a4be042b486df43905d6893fb04f05aac21c32c638e939e447/pip-9.0.1.tar.gz#md5=35f01da33009719497f01a4ba69d63c9

安装pip需要setuptools
setuptools下载：

https://pypi.python.org/packages/source/s/setuptools/setuptools-7.0.tar.gz

下载pip和setuptools放到家目录
redhat7.2自带python，如无python需要下载安装

[root@server1 ~]# ls
add.sql  demo  nginx-1.16.1.tar.gz  pip-9.0.1.tar.gz  saltapi2.py  saltapi.py  setuptools-7.0.tar.gz
[root@server1 ~]# which python
/usr/bin/python
[root@server1 ~]# python
Python 2.7.5 (default, Sep 12 2018, 05:31:16)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
[root@server1 ~]# tar zxf setuptools-7.0.tar.gz
[root@server1 ~]# ls
add.sql  demo  nginx-1.16.1.tar.gz  pip-9.0.1.tar.gz  saltapi2.py  saltapi.py  setuptools-7.0  setuptools-7.0.tar.gz
[root@server1 ~]# cd setuptools-7.0
[root@server1 setuptools-7.0]# python setup.py install[root@server1 ~]# tar zxf pip-9.0.1.tar.gz
[root@server1 ~]# cd pip-9.0.1
[root@server1 pip-9.0.1]# python setup.py install

pip 安装完成

4.Saltstack安装

这里简单示范
server1为master端，server2为minion端

[root@server1 ~]# vim /etc/yum.repos.d/saltstack.repo[saltstack-repo]
name=SaltStack repo for RHEL/CentOS $releasever
baseurl=https://repo.saltstack.com/yum/redhat/$releasever/$basearch/latest
enabled=1
gpgcheck=1
gpgkey=https://repo.saltstack.com/yum/redhat/$releasever/$basearch/latest/SALTSTACK-GPG-KEY.pub[root@server1 ~]# yum clean all
[root@server1 ~]# yum repolist
[root@server1 ~]# yum install -y salt-master##server2一样配置，只不过安装的包不一样
[root@server2 ~]# yum install -y salt-minion

接下来进行配置

[root@server2 ~]# sed -i.bak 's/#master: salt/master: server1/g' /etc/salt/minion
[root@server1 ~]# systemctl start salt-master
[root@server2 ~]# systemctl start salt-minion
[root@server1 ~]# salt-key -L
Accepted Keys:
Denied Keys:
Unaccepted Keys:
server2
Rejected Keys:
[root@server1 ~]# salt-key -A

配置成功

5.MariaDB安装

[root@server1 ~]# yum install -y mariadb-server
[root@server1 ~]# systemctl start mariadb
[root@server1 ~]# mysql        ##刚装上是无密码的，为安全，我们设个密码吧
[root@server1 ~]# mysql_secure_installation        ##出来提示除了输密码，其他均可直接回车
[root@server1 ~]# mysql -uroot -pEnter password:
安装MySQL-python模块
[root@server1 ~]# yum install -y MySQL-python

MariaDB安装完成

6.nginx环境

请参阅：
nginx安装与基本配置
简单示范
下载好nginx放到根目录

[root@server1 ~]# tar -zxf nginx-1.16.1.tar.gz
[root@server1 ~]# cd nginx-1.16.1/
[root@server1 nginx-1.16.1]# sed -i.bak 's/#define NGINX_VER          "nginx\/" NGINX_VERSION/#define NGINX_VER          "nginx"/g' src/core/nginx.h
[root@server1 nginx-1.16.1]# sed -i.bak 's/CFLAGS="$CFLAGS -g"/#CFLAGS="$CFLAGS -g"/g' auto/cc/gcc
[root@server1 nginx-1.16.1]# yum install -y gcc-c++ pcre-devel openssl-devel
[root@server1 nginx-1.16.1]# ./configure --prefix=/usr/local/nginx --with-http_ssl_module --with-file-aio --with-threads --with-http_stub_status_module
[root@server1 nginx-1.16.1]# make && make install
[root@server1 nginx-1.16.1]# ln -s /usr/local/nginx/sbin/nginx /usr/local/sbin/
[root@server1 nginx-1.16.1]# nginx -t
[root@server1 nginx-1.16.1]# nginx

网页访问测试 http://172.25.27.1/

安装成功，后续再配置

7.Salt API安装配置

[root@server1 ~]# yum install -y salt-api
[root@server1 ~]# salt-api --version
salt-api 2017.7.2 (Nitrogen)
[root@server1 ~]# cd /etc/pki/tls/private
[root@server1 private]# openssl genrsa 2048 > localhost.key
Generating RSA private key, 2048 bit long modulus
.+++
...........+++
e is 65537 (0x10001)
[root@server1 private]# cd /etc/pki/tls/certs
[root@server1 certs]# make testcert
[root@server1 certs]# vim /etc/salt/masterdefault_include: master.d/*.conf        ##注释去掉

[root@server1 ~]# useradd saltapi
[root@server1 ~]# passwd saltapi       #westos
[root@server1 ~]# vim /etc/salt/master.d/api.confrest_cherrypy:port: 8000ssl_crt: /etc/pki/tls/certs/localhost.crtssl_key: /etc/pki/tls/private/localhost.key[root@server1 ~]# vim /etc/salt/master.d/eauth.confexternal_auth:pam:saltapi:- .*- '@wheel'- '@runner'- '@jobs'[root@server1 ~]# systemctl restart salt-master
[root@server1 ~]# systemctl start salt-api

8.Django开发环境搭建

• 用pip安装(太慢的话使用源码包安装)

[root@server1 ~]# pip install django==1.11.6

#因为我们这里的报错说pip的版本太低，所以就升级了pip版本，
但是，由于网络原因，jango的下载速度很慢，
所以我们去官网下载了jango的压缩包，
然后解压压缩包进行安装

[root@foundation21 huige]# scp '/home/huige/saltstack(second)/Django-1.11.29.tar.gz' server1:
[root@server1 Django-1.11.29]# tar zxf Django-1.11.29.tar.gz
[root@server1 Django-1.11.29]# cd Django-1.11.29
[root@server1 Django-1.11.29]# python setup.py install
Installed /usr/lib/python2.7/site-packages/pytz-2020.5-py2.7.egg
Finished processing dependencies for Django==1.11.29

测试:

[root@server1 Django-1.11.29]# python
Python 2.7.5 (default, Sep 12 2018, 05:31:16)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux2
Type "help", "copyright", "credits" or "license" for more information.import djangodjango.VERSION
(1, 11, 29, u'final', 0)

[root@server1 Django-1.11.29]# cd
[root@server1 ~]# django-admin startproject HelloDjango
[root@server1 ~]# cd HelloDjango/
[root@server1 HelloDjango]# ls
HelloDjango  manage.py
[root@server1 HelloDjango]# tree .
.
├── HelloDjango
│   ├── __init__.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
└── manage.py1 directory, 5 files[root@server1 HelloDjango]# python manage.py runserver 0.0.0.0:8888
Performing system checks...System check identified no issues (0 silenced).You have 13 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): admin, auth, contenttypes, sessions.
Run 'python manage.py migrate' to apply them.January 28, 2021 - 00:59:23
Django version 1.11.29, using settings 'HelloDjango.settings'
Starting development server at http://0.0.0.0:8888/
Quit the server with CONTROL-C.
^C

[root@server1 HelloDjango]# netstat -antlp
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      2837/sshd
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      3792/master
tcp        0      0 0.0.0.0:4505            0.0.0.0:*               LISTEN      3033/salt-master Ze
tcp        0      0 0.0.0.0:4506            0.0.0.0:*               LISTEN      3039/salt-master MW
tcp        0      0 172.25.21.1:22          172.25.21.250:39110     ESTABLISHED 4378/sshd: root@pts
tcp        0      0 172.25.21.1:4505        172.25.21.3:34076       ESTABLISHED 3033/salt-master Ze
tcp        0      0 172.25.21.1:4505        172.25.21.2:49150       ESTABLISHED 3033/salt-master Ze
tcp6       0      0 :::22                   :::*                    LISTEN      2837/sshd
tcp6       0      0 ::1:25                  :::*                    LISTEN      3792/master

[root@server1 HelloDjango]# python manage.py migrate
Operations to perform:Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:Applying contenttypes.0001_initial... OKApplying auth.0001_initial... OKApplying admin.0001_initial... OKApplying admin.0002_logentry_remove_auto_add... OKApplying contenttypes.0002_remove_content_type_name... OKApplying auth.0002_alter_permission_name_max_length... OKApplying auth.0003_alter_user_email_max_length... OKApplying auth.0004_alter_user_username_opts... OKApplying auth.0005_alter_user_last_login_null... OKApplying auth.0006_require_contenttypes_0002... OKApplying auth.0007_alter_validators_add_error_messages... OKApplying auth.0008_alter_user_username_max_length... OKApplying sessions.0001_initial... OK

[root@server1 HelloDjango]# python manage.py runserver 0.0.0.0:8888
Performing system checks...System check identified no issues (0 silenced).
January 28, 2021 - 01:00:56
Django version 1.11.29, using settings 'HelloDjango.settings'
Starting development server at http://0.0.0.0:8888/
Quit the server with CONTROL-C.

确认migration文件跟数据库同步

[root@server1 oms-valor7]# python manage.py makemigrations
重置文件
[root@server1 oms-valor7]# python manage.py migrate

查看当前migration文件记录

python manage.py showmigrations

发现报错:
然后更改

[root@server1 HelloDjango]# vim HelloDjango/settings.py
ALLOWED_HOSTS = ['*']

[root@server1 HelloDjango]# python manage.py runserver 0.0.0.0:8888

[root@server1 HelloDjango]# vim HelloDjango/view.py
from django.http import HttpResponsedef hello(request):return HttpResponse("Hello Django ! ")[root@server1 HelloDjango]# vim HelloDjango/urls.pyfrom django.conf.urls import url
#from django.contrib import admin
from . import view
urlpatterns = [url(r'^hello/', view.hello),
]

[root@server1 HelloDjango]# python manage.py runserver 0.0.0.0:8888

报错:

/usr/lib/python2.7/site-packages/Django-1.11.29-py2.7.egg/django/db/backends/mysql/base.py:101: Warning: Incorrect string value: '\xE7\xAE\xA1\xE7\x90\x86...' for column 'name' at row 10return self.cursor.execute(query, args)

这个错误大概不会影响你的一些主要功能，但是在终端会显示warning的。

错误的原因比较明显，就是

        DateTimeField received a naive datetime while time zone support is active.

所谓的数据库中存储DateTimeField类型项的时候，收到一个本地的时间，但是django的时区是动态的（默认一般是US的时区）。

    修这个bug很简单，就是在settings.py中，找到，将USE_TZ的值改为False就可以了：

       # If you set this to False, Django will not use timezone-aware datetimes.USE_TZ = False

WARNINGS: ?: (mysql.W002) MySQL Strict Mode is not set for database connection ‘default’

解决方法：
复制代码

在setting.py中添加’OPTIONS’选项

[root@server1 oms_valor7]# vim oms_valor7/settings.py
DATABASES = {'default': {'ENGINE': 'django.db.backends.mysql', #设置驱动'NAME': "sunck",  #设置数据库名'USER':'root',  #设置用户名'PASSWORD':'root',  #设置数据库密码'HOST':'localhost', #设置数据库服务器的ip'PORT':'3306', #设置端口#以下解决'OPTIONS': {"init_command": "SET sql_mode='STRICT_TRANS_TABLES'"}}
}

设置mariadb的默认字符集为utf8：
2.修改字符编码

Exception Thttp://www.baidu.com/link?url=bpA49fqJzejdDRAbxTXJ14-onS51aDkT04Qk91vy31gNXAd3ohXEV5jq4A0ype: OperationalError at /accounts/login/ Exception Value: (1366, "Incorrect string value: '\\xE7\\x94\\xA8\\xE6\\x88\\xB7...' for column 'type' at row 1")

#这个报错是数据库字符集问题，如果只是实验，将之前的库删掉(否则就要将库的字符集改为utf8)，将默认字符集改为utf8，然后重新建立saltDB库，创建用户给用户授权，然后查看saltDB库的字符集是否改为utf8，然后安装xlwt表格模块，确认migration文件跟数据库同步，重置文件

如果不修改数据库的字符集为utf8，使用默认的latin1

进到maraidb中查看字符编码


mysql> show variables like '%char%';
+--------------------------+----------------------------+
| Variable_name            | Value                      |
+--------------------------+----------------------------+
| character_set_client     | utf8                       |
| character_set_connection | utf8                       |
| character_set_database   | latin1                     |
| character_set_filesystem | binary                     |
| character_set_results    | utf8                       |
| character_set_server     | latin1                     |
| character_set_system     | utf8                       |
| character_sets_dir       | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+
8 rows in set (0.00 sec)

修改 /etc/my.cnf.d/下面的两个文件，看代码示例。

在client.cnf的[client]字段里加入

default-character-set=utf8

在 server.cnf的[mysqld]字段里加入

character-set-server=utf8

重启MariaDB

sudo systemctl restart mariadbMariaDB [(none)]> show variables like '%char%';
+--------------------------+----------------------------+
| Variable_name            | Value                      |
+--------------------------+----------------------------+
| character_set_client     | utf8                       |
| character_set_connection | utf8                       |
| character_set_database   | utf8                       |
| character_set_filesystem | binary                     |
| character_set_results    | utf8                       |
| character_set_server     | utf8                       |
| character_set_system     | utf8                       |
| character_sets_dir       | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+
8 rows in set (0.00 sec)

[root@server1 oms-valor7]# mysql -p
MariaDB [(none)]> CREATE DATABASE saltDB;
Query OK, 1 row affected (0.00 sec)MariaDB [(none)]> GRANT ALL ON saltDB.* TO django@'localhost' IDENTIFIED BY 'redhat';MariaDB [saltDB]> show create database saltDB;
+----------+-----------------------------------------------------------------+
| Database | Create Database                                                 |
+----------+-----------------------------------------------------------------+
| saltDB   | CREATE DATABASE `saltDB` /*!40100 DEFAULT CHARACTER SET utf8 */ |
+----------+-----------------------------------------------------------------+
1 row in set (0.00 sec)

[root@server1 oms-valor7]# pip install xlwt
[root@server1 oms-valor7]# python manage.py makemigrations
[root@server1 oms-valor7]# python manage.py migrate

登录MariaDB数据库验证数据库同步

[root@server1 oms-valor7]# mysql -h127.0.0.1 -udjango -p
MariaDB [(none)]> use saltDB;
MariaDB [saltDB]> show tables;
+--------------------------------+
| Tables_in_saltDB               |
+--------------------------------+
| asset_areas                    |
| asset_cities                   |
| asset_idcasset                 |
| asset_provinces                |

同步成功，创建应用的超级登录用户

[root@server1 oms-valor7]# python manage.py createsuperuser
huige
huige@westos.org
zxcvbnm23[root@server1 oms-valor7]# python manage.py runserver 0.0.0.0:9999

打开浏览器:
使用刚才创建的huige用户登陆

这里搜索不到server2

[root@server1 ~]# cd oms_valor7/[root@server1 oms_valor7]# find . -name saltapi.py
./deploy/saltapi.py[root@server1 oms_valor7]# cd deploy/
admin.py   forms.py     __init__.pyc  migrations  saltapi.py   tar_file.pyc  views.py
admin.pyc  forms.pyc    md5.py        models.py   saltapi.pyc  templatetags  views.pyc
apps.py    __init__.py  md5.pyc       models.pyc  tar_file.py  tests.py

我们将saltapi.py中的用户名和密码改为我们自己设置的(saltapi和westos)

[root@server1 deploy]# systemctl restart salt-api

然后我们点击刷新salt主机列表:

这个报错是因为ssl认证的问题，所以我们给saltapi.py中倒入ssl模块，
让其忽略ssl认证:
在全局添加如下代码：

[root@server1 deploy]# vim saltapi.py
import ssl
ssl._create_default_https_context = ssl._create_unverified_context

再次刷新主机列表时报了新的错误:

可以看到这次是认证问题，说明我们的saltapi还没有改对:
发现是saltapi.py中def init(self,url,username,password)这个函数中的username和password跟自己创建的用户密码
我们进行修改:

[root@server1 deploy]# vim saltapi.py
class SaltAPI(object):__token_id = ''def __init__(self,url,username,password):self.__url = url.rstrip('/')self.__user = 'saltapi'     #usernameself.__password = 'westos'    #password

再次刷新salt主机列表:

OMS在线运维管理平台到这里就搭建就完成了，接下来整合NGINX

二、部署进阶

1.配置Nginx支持Django
1.安装uwsgi

[root@server1 ~]# pip install uwsgi##如果出现以下错误：plugins/python/uwsgi_python.h:2:20: fatal error: Python.h: No such file or directory#include <Python.h>^compilation terminated.----------------------------------------
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-cj_mjG/uwsgi/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-sHka7Q-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-cj_mjG/uwsgi/

[root@server1 ~]# yum install python-devel
[root@server1 ~]# pip install uwsgi        ##安装python-devel之后再安装就ok了
Collecting uwsgiUsing cached uwsgi-2.0.19.1.tar.gz
Successfully installed uwsgi-2.0.19.1
[root@server1 ~]# uwsgi --version  ##查看版本2.0.19.1

测试uwsgi的可用性

[root@server1 ~]# vim uwsgi.pydef application(env, start_response):start_response('200 OK', [('Content-Type','text/html')])return "Hello Uwsgi"
[root@server1 ~]# uwsgi --http :8888 --wsgi-file uwsgi.py

2.配置NGINX支持uwsgi

Django结合了nginx之后就不需要再执行runserver来启动服务了，而且访问端口就是nginx的默认端口80，使用时方便许多

[root@server1 ~]# vim /usr/local/nginx/conf/nginx.conf
user  root;server {listen       80;server_name  localhost;#charset koi8-r;root /root/oms_valor7;#access_log  logs/host.access.log  main;access_log  logs/host.access.log;location / {root   /root/oms_valor7;include uwsgi_params;uwsgi_pass 127.0.0.1:9999;uwsgi_param UWSGI_CHDIR /root/oms_valor7;uwsgi_param UWSGI_SCRIPT oms_valor7.wsgi;}
location /static {alias /root/oms_valor7/static;}

3.uwsgi 配置

uwsgi支持ini、xml等多种配置方式，本文以 ini 为例，在/root/oms_valor7/目录下新建uwsgi.ini，添加如下配置

[uwsgi]
chdir=/root/oms-valor7/
module=oms_valor7.wsgi:application
master=True
pidfile=/var/run/uwsgi9999.pid
vacuum=True
max-requests=5000
daemonize=/root/oms_valor7/uwsgi9999.log
socket=0.0.0.0:9999

4.启动服务，运行平台

重载nginx ，并将uwsgi后台运行

[root@server1 oms-valor7]# cd
[root@server1 oms-valor7]# nginx -t
[root@server1 oms-valor7]# nginx -s reload
[root@server1 oms-valor7]# uwsgi --ini uwsgi.ini &

在浏览器中直接访问IP:

点击远程命令，输入自己要执行的远程命令

看来不能直接运行，要选择主机

三、参考文章

本文原参考文章连接

本文只是根据自己的环境照着原文来搭建，因为环境不同，
遇到很多和原文不同报错，最终解决，这里分享排错思路，希望可以帮到大家