1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
|
.. _internals_l2_isolation:
****************
Ryu L2 isolation
****************
This section describes how Ryu cooperates with Openstack Quantum and
how its L2 isolation works.
Overview
========
Ryu provides REST API by which Quantum server tells necessary informations.
Quantum Server manages the association networks(uuid) to actual key value in
addition to normal Quantum management informations.
(Here key value is an integer for VLAN ID, GRE key and so on.
The quantum only have to know the range of key which depends on the isolation
technology. For example, 12 bit in VLAN case, 24 bit in GRE case.)
Quantum Ryu plugin doesn't know about what technology Ryu uses for L2
isolation.
.. image:: /images/internal-quantum-overview.png
Quantum doesn't necessarily knows all the informations Ryu needs like
mac address attached to the interface. Ryu can gather those informations
by accessing directly to OVSDB. When tunnel ports on OVS needs to be created
on compute-node, Ryu will directly accesses to OVSDB and creates/deletes
ports.
Cooperate with Openstack Quantum
================================
Ryu reacts to Qauntnum events, compute-node boots up, network
creation/deletion, and VM instance creation/deletion.
When VM instance is created, corresponding quantum port is created.
compute-node boot up
--------------------
When a compute note boots up, minimal initalization work is done by
Ryu-quantum-agent which tell necessary informations to Ryu.
Then Ryu set up OVS such that OVS connects to Ryu via OpenFlow.
There are 2 steps of OVS initialization. By agent and by Ryu.
This is to keep Ryu agent logic minimal and independent from what actual Ryu
needs to set. Even if Ryu is enhanced for new feature and some additional
configuration to OVS is needed (for example multi-controller for HA),
ryu agent doesn't need to be modified due to 2 step initialization.
.. image:: /images/internal-quantum-bootup.png
network creation
----------------
When network is created, quantum Ryu plugin assigns Key value to
a created network, and tell the association to Ryu
.. image:: /images/internal-quantum-network-creation.png
VM instance creation
--------------------
When VM instance is created, quantum port is created. Quantum Ryu
plugin tells the association of (network uuid, port uuid) to Ryu, and
then OVS port is created. Ryu finds the port creation via OpenFlow
and get the information of the created port for (port uuid, attached
mac address) via OVSDB protocoal, and then sets up network
configuration to OVS.
.. image:: /images/internal-quantum-instance-create.png
quantum_adapater RyuApp
-----------------------
This application watches port creation/deletion by OF protocol.
When it detects the creation of ports, it tries to retrieve related
informations(port uuid, mac address) by OVSDB protocol,
tries to determine if the port corresponds to Qauntnum VM port, and then
stores those informations into the in-memory, which generates the event of
VMPort creation. Then Ryu app of isolation (simple_vlan or gre_tunnel)
will be notified.
live-migration
--------------
live-migration is popular feature with virtualization, so as OpenStack.
As of this writing, there is no hooks in quantum. So no notification/callback
are triggered when live-migration starts/on-going/ends/error-abort.
Traditional live-migration uses GARP to tell switches the used mac address
is moved.
.. image:: /images/internal-live-migration.png
VLAN
====
OVS supports port vlan with setting tag value in OVSDB.
Ryu utilizes it for L2 isolation.
simple_vlan RyuApp
------------------
When port is created, it sets tag value to key assigned to a given network uuid.
And sets flow entry to output:normal.
live-migration
--------------
As flows includes output:normal action, packets are processed by
OVS builtin mac-learning.
#. When destination VM port is created, same rule is inserted on OVS
on the destination.
But the port on the destination is not used until the first GARP packet
is sent
#. When VM is resumed on the destination, a GARP packet is sent.
Then, mac learning tables on each switch are updated.
So the port on the source will be unused.
#. When the VM on the source is destroyed, the port on the source is also
destroyed.
GRE tunneling
=============
OVS supports tunneling and Ryu utilizes it for L2 isolation as follows.
.. image:: /images/internal-gre-tunnel.png
tunnel_port_updator RyuApp
--------------------------
This application watches the VM port creation/deletion, and creates/deletes
tunnel port on OVS when necessary.
That is, it creates tunnel port between compute-nodes which have VMs of same
tenant. it deletes tunnel ports when compute-nodes have no VMs of same
tenant.
gre_tunnel RyuApp
-----------------
This application watches VM/tunnel port creation/deletion, and
installs/removes flow entries based on port creation/deletion.
Flow Entries
------------
Ryu installs following flow entries.
.. image:: /images/internal-quantum-gre-flow-table.png
live-migration
--------------
As flows are aware of mac address of each ports, Ryu updates flow table
for live-migration on each compute-nodes.
When the port of same mac address is added on another compute-node,
Ryu installs flows that duplicates packet so that packets destined to
the mac address will be duplicated and sent to both ports.
GARP from hypervisor isn't used.
.. image:: /images/internal-tunnel-live-migration-before.png
.. image:: /images/internal-tunnel-live-migration-during.png
.. image:: /images/internal-tunnel-live-migration-after.png
Mac address based L2 isolation
==============================
Ryu also supports mac address based L2 isolation.
In this case key isn't used.
mac learing
-----------
When VM sends packets, Ryu determins network uuid from OVS port and then
associates src mac address to network uuid.
.. image:: /images/mac-learning.png
packet filtering(L2 unicast case)
---------------------------------
* When VM sending L2-unicast packet, Ryu checks if the destination mac
address belongs to the same netowrk id of the source mac address which
is same to the network uuid that the OVS port is associated to.
* If no, the packet is dropped.
* If yes, send the packet is sent to ports which belongs to the same
network uuid and external port.
.. image:: /images/filtering-outgoing.png
.. image:: /images/filtering-incoming.png
packet filtering(L2 broadcast case)
-----------------------------------
* When VM sending L2-broadcast/multicaset packet, Ryu checks if the source
mac address.
* send the packet to all external ports and all OVS ports that belongs
to the same network uuid of the source mac address.
* When receiving broacast/multicast packet from the external ports,
Ryu checks if the source mac address belongs to known network uuid.
* If yes, send the packet to the external ports except incoming one
and the all OVS ports that belongs to the network uuid
* if no, drop the packet.
.. image:: /images/filtering-broadcast.png
live-migration
--------------
As of this writing, simple isolation doesn't support live-migration.
|
