Suning Cloud Commerce is one of the largest privately owned retailers in China. Suning has more than 1600 stores covering over 700 cities of Mainland China, Hong Kong and Japan, and its e-commerce platform, Suning.com ranks among top three Chinese B2C companies. There are more than 180,000 employees, thousands of mixed power, x86, storage servers and tens of thousands of virtual machines from several large data center across China, HongKong and Japan. KVM, oVirt and virtualization technologies are widely used, and there are also very large server farm for VDI.
Till end of year 2014, Suning has setup large OpenStack
private production clouds across several data centers, based on OpenStack
Icehouse. Controller nodes are
high-availabile and easily scale-out based on Pacemaker+Corosync+HAproxy, with large compute+storage nodes, splitted by multiple
regions, and each region was further splitted into multiple
availability zones.
Host aggregates are also used with pre-determined
metadata attributes to serve complex scheduling not only based on CPU, Memory, Disk, but also filters like self-developed
anti-Affinity on anti-Affinity according to business requirement.
Config drive is used for the isolated AZ that can only accept static IP address. iSCSI burden is also tweaked to fast
Cinder volume to instances to improve IO performance.
Security is a forever topic for any IT infrastructure, especially important in a large production OpenStack cloud, which involving:
- Operating System Level Security Enforcement and intrusion detection;
- Password Security, especially Host and Virtual Machine password, life cycle from template creation to virtual machine retirement;
- Message level protection including message routing from generation to consumption;
- Database security settings to prevent unauthorized access or privilege alter;
- VNC/Spice console protection;
- Service port restriction;
- Network DDoS attack detection;
- Account, Password and ssh key management;
- Openstack service protocol protection;
- Virtual Machine access and isolation along physical planning;
In this presentation, we will share approaches that we utilize in setup large OpenStack cloud
