未命名

发表于 2017-05-23 |

molecule ansible 自动测试工具

文档： https://molecule.readthedocs.io

未命名

发表于 2017-05-23 |

ansible template 模块讲解

如果在playbook中使用 template命令的话

ansible会在 ansible runner.py 文件中如下方式调用 _execute_template 函数

注：至少在v0.0.2 版本的 template命令依赖于客户端需要jinja2 支持

runner.py 版本v0.0.2 -------------
       elif self.module_name == 'template':
           result = self._execute_template(conn, host, tmp)
                    -------------
   def _execute_template(self, conn, host, tmp):
       ''' handler for template operations '''
       # load up options
       options  = self._parse_kv(self.module_args)
       source   = options['src']
       dest     = options['dest']
       metadata = options.get('metadata', None)
       if metadata is None:
           if self.remote_user == 'root':
               metadata = '/etc/ansible/setup'
           else:
               metadata = '~/.ansible/setup'
       # first copy the source template over
       tpath = tmp## 随机路径 比如:~/.ansible/xxx_temp_1234/
       tempname = os.path.split(source)[-1]  #源文件 比如 nginx.conf.j2
       temppath = tpath + tempname  #组合后 为：~/.ansible/xxx_temp_1234/nginx.conf.j2
       self._transfer_file(conn, utils.path_dwim(self.basedir, source), temppath)
       # install the template module
       template_module = self._transfer_module(conn, tmp, 'template')
       # run the template module
       #先执行 template_module  
       args = [ "src=%s" % temppath, "dest=%s" % dest, "metadata=%s" % metadata ]
       (result1, executed) = self._execute_module(conn, tmp, template_module, args)
       results1 = self._return_from_module(conn, host, result1, executed)
       (host, ok, data) = results1
       ###如果 template module 执行正常，再调用file，模块修改相关权限
       # magically chain into the file module
       if ok:
           # unless failed, run the file module to adjust file aspects
           old_changed = data.get('changed', False)
           module = self._transfer_module(conn, tmp, 'file')
           args = [ "%s=%s" % (k,v) for (k,v) in options.items() ]
           (result2, executed2) = self._execute_module(conn, tmp, module, args)
           results2 = self._return_from_module(conn, host, result2, executed)
           (host, ok, data2) = results2
           new_changed = data2.get('changed', False)
           data.update(data2)
           if old_changed or new_changed:
               data['changed'] = True
           return (host, ok, data)
       else:
           # copy failed, return orig result without going through 'file' module
           return results1

未命名

发表于 2017-05-23 |

aaa 集群环境

整体结构图

                            +                                
        +--------+          |            +------+            
        |  OSS1  |          |            | OSS2 |            
        +---+----+          |            +---+--+            
            |               |                |               
            |               v                |               
            +--keepalive--+ vip +--kippalive-+               
                            |                                
                            |                                
                            +                                
            +-haproxy+---+rabbitmq:5670 +-haproxy-+          
            |               +                     |          
+----------+--------+       |          +----------+---------+
|rabbitmq_node1:5672|       |          |rabbitmq_node2：5672|
+-------------------+       |          +--------------------+
                            v                                
                +------------+-----------+                   
                |      rabbitmq队列       |                   
                +------------+-----------+                   
                            |                                
                            |                                
    keepalive+haproxy       |          keeplaive+haproxy     
                            v                                
            +--------+ AAA_vip:6060+---------+               
            |               +                |               
            |               |                |               
    +-----+-------+         |         +------+-------+       
    |  aaa1:6600  |         |         |  aaa2:6600   |       
    +-------------+         |         +--------------+       
                            |                                
                            v

aaa集群

这里以两机集群演示，4机等类似

k:keepalive

h:haproxy

           |
         机顶盒
           |
           v
        vip:6060  -> vip 这里是keepalive设置的 virtual ip
          /   \
         /     \
        /       \
aaa1:6600(k+h)  aaa2:6600(k+h)

keepalive: 设置vip 每台主机的权值依次递减，正常情况下vip会一直存在于aaa1 主机

haproxy: 所有主机配置一致，当vip的6060收到请求，会自动转发到aaaX的6600端口

aaa数据库同步:
每个aaa的数据是独立存放在自己的数据库中，数据来源于rabbitmq，每台主机创建一条唯一的队列，注册到对应exchange

oss 发广播到 rabbitmq的exchange，各自aaa消费自己对应的队列，把数据入库

这种架构，能保证用户的随机读，也可保证写数据一致，代码中不用区分集群和非集群环境。

oss 和 aaa的通信

oss不直接和aaa进行通信,rabbitmp 作为一个消息队列，是传输的中间层

       |
      oss
       |
       v
+------+-----+
|rabbitmp队列 |
+------------+
    /   \
   /     \
  /       \
aaa1       aaa2

aaa 中rabbitmq集群

oss:
作为消息的生产者
发广播到指定的oss_exchange

aaa:
队列创建者：aaa，注入到指定的exchange 消费者：aaa
创建队列时，会有个partion值，这个一般为主机名，这样就可以n个主机，n条队列，各自消费自己

注：aaa节点机不需要部署rabbitmq

aaa 中rabbitmq集群的高可用
利用haproxy，侦听 5670，转发到node：
```
- rabitmq_cluser:5670
     - rabitmp_node1:5672
     - rabitmp_node2:5672
```
每台rabbitmq主机，都安装haproxy，做这样的配置
rabbitmq数据同步： rabbitmq集群主机会在各节点间自动同步，rabitmp_node1和rabitmp_node2组建集群即可

未命名

发表于 2017-05-23 |

ansible开启代理 + pipline的步骤

chmod u+s /usr/bin/sudo
vi /etc/sudoers 删除
Defaults requiretty
vi /etc/sudoers 添加
fonsview ALL=(ALL) ALL

下面是ansible自动搞定

先使用su 变为root
开启sudo
使用sudo干活


-- hosts
[gatewayed]
test2 ansible_host=10.x.x.x	ansible_user=fonsview	ansible_ssh_pass=xxxxx  ansible_port=50000  proxy_ip=219.x.x.x
-- group_vars/gatewayed
#ansible_ssh_common_args: '-o ProxyCommand="ssh -p 50000 -W %h:%p -i ~/.ssh/id_rsa.pub fonsview@{{ proxy_ip }}"'
---
- hosts: gatewayed
  become: true
  become_user: root
  become_method: sudo
  # remote_user: root
  gather_facts: no
  vars:
      ansible_ssh_pipelining: true
      ansible_sudo_user: "{{ ansible_user }}" 
      ansible_sudo_pass: "{{ ansible_ssh_pass }}"
  tasks:
  - file: path=/usr/bin/sudo  mode=u+s 
    become_method: su
    vars:
        ansible_ssh_pipelining: no
        ansible_become_pass: "AHxxx*"
  - lineinfile:
      dest: /etc/sudoers
      regexp: '^Defaults    requiretty'
      line: '#Defaults    requiretty'
      state: present
    become_method: su
    vars:
        ansible_ssh_pipelining: no
        ansible_become_pass: "AHxxx"
  - lineinfile: "dest=/etc/sudoers state=present regexp='^fonsview' line='fonsview  ALL=(ALL)       ALL'"
    become_method: su
    vars:
        ansible_ssh_pipelining: no
        ansible_become_pass: "AHxxx"
  - name: "查询我是哪个用户"
    command: whoami
    register: xx
  - debug: var=xx
  - name: "创建一个文件"
    copy: src=site.yml dest=/tmp/ntp2.conf

未命名

发表于 2017-05-23 |

centos7 配置使用阿里的yum源-使用curl

创建备份目录
mkdir -p /etc/yum.repos.d/bak
把老的配置移到备份目录
mv /etc/yum.repos.d/*.repo /etc/yum.repos.d/bak/
清楚yum缓存
yum clean all
下载yum源配置

curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
curl -o /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo

重新生成缓存
yum makecache
下载软件包到本地
mkdir /tmp/ansible

yum install ansible –downloadonly –downloaddir=/tmp/ansible/

centos6 配置使用阿里的yum源

创建备份目录
mkdir -p /etc/yum.repos.d/bak
把老的配置移到备份目录
mv /etc/yum.repos.d/*.repo /etc/yum.repos.d/bak/
清楚yum缓存
yum clean all
下载yum源配置

curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-6.repo
curl -o /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-6.repo

重新生成缓存
yum makecache
下载软件包到本地
mkdir /tmp/ansible

yum install ansible –downloadonly –downloaddir=/tmp/ansible/

未命名

发表于 2017-05-23 |

cms 注入流程

添加contenID 管理

1. 系统配置 ->  contenID管理 -> 添加 ->

                        - 批次： 1234
                        - contenID预分配数量： 200
                        - 执行时间 -> 选当前时间
2. CP/SP -> CP -> 添加 -> （注：CP ID与CMM设置中的CMSID要一致对应）

                        - CP ID: fonsview_hls
                        - 名称*: fonsview_hls
                        - CP内容编号*: 00000001
3. CP/SP -> SP -> 添加 ->

                        - SP ID*: fonsview_hls
                        - 名称*: fonsview_hls
                        - CP内容编号*: 00000002
                        - 内容提供商: 全选
4. 添加角色 -> 

            - 内容名称： monkey
            - 描述: monkey test
            - 设置权限： 全选
5. 添加用户 -> 选择 -> CP
                        - CP: fonsview_hls
                        - 登录帐号: monkey

6. 系统管理 -> FTP服务器 -> 添加/修改 ->

                                    - 服务器名称: ott-ftp
                                    - 主机名或IP: 172.16.199.119
                                    - 端口: 21
                                    - 用户名: ftpuser
                                    - 密码: ftpuser
                                    - 根目录: /opt/fonsview/data/media     (注:ftp那里配置好chroot后,这里根即为/opt/fonsview/data/media)

CDN 分发

1. 分发管理 -> 分发配置 -> CDN分发 ->

                      - 配置名称*: cd
                      - 接口类型: CD
                      - 启用: 是
                      - 流量控制: 10
                      - 超时时长: 60
                      - 分发地址: http://172.16.199.120:8190/ContentDeployReq   (CMM地址)
                      - FTP服务器地址: ott-ftp
                      - 播放信息:  (如下都为SS地址)
                        - RR_IPTV  rtsp://172.16.199.120:554
                        - RR_HPD   http://172.16.199.120:8106
                        - RR_HLS   http://172.16.199.120:8112

VOD 注入

1. 点播放管理 -> 合集列表 -> 添加 -> 添加合集
2. 点播放管理 -> 合集列表 -> 列表 -> 在刚才添加的合集上 -选 操作 -> 分集管理
                        - 添加 片源  
                        - 链接: ftp://anonymous:anonymous@172.16.200.250/Media/1flora/vod/qianlizhiwai/index.m3u8

日志查看:

cms 的log

cms 接收到用户的注入信息

[root@fonsview log]# cat cms.log | grep “MediaContentInterface”
02/12/2016 17:01:12 http-bio-6600-exec-8 INFO MediaContentInterface:137 - received json is:{“id”:0,”createDate”:”2016/12/02 17:01:12”,”validTime”:null,
“hdPrice”:0.0,”price”:0.0,”flag”:0,”downloadUrl”:”ftp://anonymous:anonymous@172.16.200.250/Media/1flora/vod/qianlizhiwai/index.m3u8”
…
添加好分集
…
02/12/2016 17:01:12 http-bio-6600-exec-8 INFO MediaContentInterface:170 - add mediaContent successfully
02/12/2016 17:01:12 http-bio-6600-exec-8 INFO MediaContentInterface:962 - mediacontent id is:2
cms 创建REGIST xml 文件
02/12/2016 18:37:00 schedulerFactoryBean_Worker-10 DEBUG Create2CdnXmlFile:284 - begin create movie REGIST xml file
02/12/2016 18:37:00 schedulerFactoryBean_Worker-10 INFO SendCdMessage:137 - CMSID=mango_hls,SOPID=cd,CorrelateID=40b56b8e424845028c5ae1b1af956218,ContentMngXMLURL=ftp://ftpuser:ftpuser@172.16.199.119:21/CDN/CD/mango_hls/20161202/Movie_REGIST_mango_hls_4_20161202183700_020289.xml
cmm 下载 xml (cmm log)
2016-12-05 19:21:49,041 INFO CMS distribute info:CMSID[Mango],SOPID[cd],CorrelateID[bbb81a4232ae44759e5d9266f9f6077d],ContentMngXMLURL[ftp://ftpuser:ftpuser@172.16.199.120:21/CDN/CD/Mango/20161205/Movie_UPDATE_Mango_1_20161205192149_909034.xml] [ContentDeployReqService.java:35]
2016-12-05 19:21:49,042 INFO Insert CD request to DB, call procedure SpContentDeployReqAdd(Mango,cd,bbb81a4232ae44759e5d9266f9f6077d,ftp://ftpuser:ftpuser@172.16.199.120:21/CDN/CD/Mango/20161205/Movie_UPDATE_Mango_1_20161205192149_909034.xml) [MessageQueueManager.java:66]
2016-12-05 19:21:49,044 INFO Insert CD request to DB success [MessageQueueManager.java:78]
2016-12-05 19:21:49,044 INFO addRequestMessage rlt = true [ContentDeployReqService.java:58]
2016-12-05 19:21:49,044 INFO CMM receive success and return [ContentDeployReqService.java:70]
2016-12-05 19:21:49,712 INFO parser thread:request info—>CMSID:Mango,SOPID:cd,ftpaddress:ftp://ftpuser:ftpuser@172.16.199.120:21/CDN/CD/Mango/20161205/Movie_UPDATE_Mango_1_20161205192149_909034.xml,CorrelateID:bbb81a4232ae44759e5d9266f9f6077d [XmlParserThread.java:47]
2016-12-05 19:21:49,712 INFO getContentMngXMLURLftp://ftpuser:ftpuser@172.16.199.120:21/CDN/CD/Mango/20161205/Movie_UPDATE_Mango_1_20161205192149_909034.xml [XmlParserThread.java:67]
2016-12-05 19:21:49,712 INFO download:ftp://ftpuser:ftpuser@172.16.199.120:21/CDN/CD/Mango/20161205/Movie_UPDATE_Mango_1_20161205192149_909034.xml [FtpUtils.java:32]
2016-12-05 19:21:49,713 INFO download try:ftp://ftpuser:ftpuser@172.16.199.120:21/CDN/CD/Mango/20161205/Movie_UPDATE_Mango_1_20161205192149_909034.xml [FtpUtils.java:40]
2016-12-05 19:21:49,715 INFO connect try:21 [FtpUtils.java:42]
2016-12-05 19:21:49,719 INFO login try:ftpuser,ftpuser [FtpUtils.java:44]
2016-12-05 19:21:49,719 INFO true [FtpUtils.java:49]
2016-12-05 19:21:49,719 INFO 257 “/“
[FtpUtils.java:53]
2016-12-05 19:21:49,719 INFO /CDN/CD/Mango/20161205 [FtpUtils.java:54]
2016-12-05 19:21:49,777 INFO CD type:Movie [AdiXmlParser.java:65]
2016-12-05 19:21:49,778 INFO FileURL: ftp://anonymous:anonymous@172.16.200.250/Media/1flora/vod/qianlizhiwai/index.m3u8 [AdiXmlParser.java:529]
2016-12-05 19:21:49,778 INFO CPContentID: Mango [AdiXmlParser.java:529]
2016-12-05 19:21:49,778 INFO Duration: 0 [AdiXmlParser.java:529]
2016-12-05 19:21:49,778 INFO FileSize: [AdiXmlParser.java:529]
2016-12-05 19:21:49,779 ERROR can not transfor to int value [StringUtils.java:47]
2016-12-05 19:21:49,779 INFO Domain: 1032 [AdiXmlParser.java:529]
2016-12-05 19:21:49,779 INFO DestDRMType: 0 [AdiXmlParser.java:529]
2016-12-05 19:21:49,783 INFO [null_00000002000000001234000000000007]add MovieInjectReq success [MovieInjectQueue.java:31]
2016-12-05 19:21:49,785 INFO delete CD request:reqId[919] from db success [Procedure.java:137]
ss 收到cmm 传来的信息
20161206 14:32:10.255.442 dec11 (a593dcd4791f8d77)(0)DECI0 got GET msg from 0.0.0.0,start=0,copies=1,url=ftp://anonymous:anonymous@172.16.200.250/Media/1flora/vod/qianlizhiwai/index.m3u8,pcontent_id=00000002000000001234000000000011,provider_id=fonsview_hls,OriginProviderID=,OriginContentID=,file_type=9[deci11_fsm_msg.c:322]
ss 拉取源信息

20161206 14:32:11.960.729 demi11 [a593dcd4791f8d77, 0,1006,-1]encode req len=47 url=SIZE Media/1flora/vod/qianlizhiwai/index.m3u8

ss 拉取完成返回给 cmm
20161206 14:32:54.429.795 dec11 (a593dcd4791f8d77)reply complete msg to 172.16.199.120:8194,pcontent_id:00000002000000001234000000000011,provider_id:fonsview_hls,retcode=200[deci11_fsm_msg.c:1636]
cmm 收到 ss的返回信息
[ContentDistributeRes.java:69]
2016-12-06 14:32:10,264 INFO [fonsview_hls_00000002000000001234000000000011]Send Movie[00000002000000001234000000000011] to CDN success [MovieSessionThread.java:610]
2016-12-06 14:32:54,434 INFO ss request uri:/OnlineNotifyReq [HttpHandlerImpl.java:794]
2016-12-06 14:32:54,435 INFO /172.16.199.120: contentmgr request info:<?xml version=”1.0” encoding=”UTF-8”?>
cmm 将xml 文件上传到自己的ftp (注这里是cmm的ftp,也可以和cms共用ftp)

2016-12-06 14:32:56,206 INFO Ftp: ftp://ftpuser:ftpuser@127.0.0.1/cmm/Mango_cd_e124f5c8a3e84b90843cd4d30958e767_1481005976170.xml [FtpUtils.java:317]
2016-12-06 14:32:56,207 INFO ftp: 127.0.0.1:21 [FtpUtils.java:331]
2016-12-06 14:32:56,209 INFO ftp: ftpuser:ftpuser [FtpUtils.java:336]
2016-12-06 14:32:56,214 INFO ftp Reply: 230 [FtpUtils.java:341]
2016-12-06 14:32:56,216 INFO upload ftp successfully. [FtpUtils.java:403]
2016-12-06 14:32:56,217 INFO cd Result xml upload end … [WsCdClientThread.java:67]

cms 接受到到 cmm传来的消息
06/12/2016 14:32:56 qtp596155362-37 INFO ReplyCD:75 - received message from CMM: CMSID=Mango, SOPID=cd, CorrelateID=e124f5c8a3e84b90843cd4d30958e767, ResultCode=0, ErrorDescription=null, ContentMngXMLURL=ftp://ftpuser:ftpuser@127.0.0.1/cmm/Mango_cd_e124f5c8a3e84b90843cd4d30958e767_1481005976170.xml
06/12/2016 14:32:56 qtp596155362-37 DEBUG ReplyCD:117 - >>>>>process cdn task sucessfully by cmm correlateId=e124f5c8a3e84b90843cd4d30958e767, contentId=00000002000000001234000000000011

未命名

发表于 2017-05-23 |

##检测数据同步

cms和oss同步检测
分发管理 –> 分发配置 –> BMS

- 配置名称: 111
- 接口类型: CB-
- 启用: 是
- 流量控制: 200
- 超时时长: 60
- 分发地址: http://172.16.199.120:8081/oss/rest/syncContent
- FTP服务器: ott-ftp
- 重试次数: 1
- CP: fonsview_hls
- 内容类型: 全选

添加之后
在分发管理 –> 分发历史 –> BMS分发
可以看到很多内容

这时打开 oss的管理地址,在
产品管理 –> 内容能看到有同样的信息

oss和aaa同步检测
说明 cms 和 oss 同步没有问题.

这时也可以在oss中创建一些账户,在cmsdb中的account表中也能看到相应的账户,说明oss和aaa同步数据成功.

模板导入

CMS -> EPG管理 – > 首页模板
```
-导入首页模板 --> 导入xml
```
CMS -> EPG管理 – > 首页配置
版本控制 -> 添加 -上传apk
版本列表 –> 应用程序 –> 关联

盒子测试地址

8个上键进入设置地址
输入密码 fs123

aaa地址:http://172.16.199.120:6600/aaa/services/rest

epg地址:http://172.16.199.120:6600/epg/rest/SPM

oss 创建账户,绑定机顶盒mac地址

在oss 中创建账户
关联机顶盒
选中这个用户点更换机顶盒 –从未关联的设备中,选择你的设备,进行关联

aaa 日志中会有这样的信息,表明用户登录成功
07/12/2016 19:24:30 DEBUG AppService:230 - d8c1312a-968f-486f-b201-9107a8ebaf57 user login success ,userId:U00000013 ,fuserId:U00000013,userToken:7904E212D9D7D6F0ABF53D2ABBBFBC55

oss 中添加epg地址

oss中资源配置 –> 添加 -> -名称 epg
-服务类型 epg地址

- 服务器域名:http://172.16.199.120:6600/epg/rest/SPM

清除epg缓存

http://172.16.199.120:6600/epg/rest/SPM/V2/clearcache

未命名

发表于 2017-05-23 |

#easysnmp

dnf install net-snmp-devel
pip install easysnmp

from easysnmp import Session

Create an SNMP session to be used for all our requests

session = Session(hostname=’demo.snmplabs.com’, community=’public’, version=2)

You may retrieve an individual OID using an SNMP GET

location = session.get(‘sysLocation.0’)

You may also specify the OID as a tuple (name, index)

Note: the index is specified as a string as it can be of other types than

just a regular integer

contact = session.get((‘sysContact’, ‘0’))

And of course, you may use the numeric OID too

description = session.get(‘.1.3.6.1.2.1.1.1.0’)

Set a variable using an SNMP SET

session.set(‘sysLocation.0’, ‘The SNMP Lab’)

Perform an SNMP walk

system_items = session.walk(‘system’)

but also has several extended attributes with SNMP-specific information

for item in system_items:
print ‘{oid}.{oid_index} {snmp_type} = {value}’.format(
oid=item.oid,
oid_index=item.oid_index,
snmp_type=item.snmp_type,
value=item.value
)

未命名

发表于 2017-05-23 |

curl -XGET ‘localhost:9200/_cat/health?v&pretty’

可以查询到集群的状态节点数

epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1475871424 16:17:04 elasticsearch green 1 1 5 5 0 0 0 0 - 100.0%

未命名

发表于 2017-05-23 |

部署容器化
集中关注持续运行的无状态应用

pod
- 资源共享与通信
- 单独的pod总体来说不适合拿来运行同一应用的多个实例。

molecule ansible 自动测试工具

ansible template 模块讲解

aaa 集群环境

ansible开启 代理 + pipline的步骤

cms 注入流程

添加contenID 管理

CDN 分发

VOD 注入

日志查看:

模板导入

盒子 测试地址

oss 创建账户,绑定机顶盒mac地址

oss 中添加epg地址

清除epg缓存

Create an SNMP session to be used for all our requests

You may retrieve an individual OID using an SNMP GET

You may also specify the OID as a tuple (name, index)

Note: the index is specified as a string as it can be of other types than

just a regular integer

And of course, you may use the numeric OID too

Set a variable using an SNMP SET

Perform an SNMP walk

Each returned item can be used normally as its related type (str or int)

but also has several extended attributes with SNMP-specific information

ansible开启代理 + pipline的步骤

盒子测试地址