Skip to content

ipgallery / devops

Go to a project
Commit d8cab095 by Amir Aharon
Options

changes in design and multi-instance

parent 9d0495b8
Inline Side-by-side
Showing with 112 additions and 18 deletions
common/docs/knowledgeBase/MultiInstance-Configuration.txt
CONFIGURING MULTI-INSTANCE FOR MCX
- First we need to enable private ips on all instances:
https://www.digitalocean.com/docs/networking/vpc/how-to/enable/
This way we don't pay for the traffic among those instances as long as they communicate via their
private ips.
The instances must be under the same VPC (Virtual ,private connection) to receive the same ip range and rules.
- Altough we have private ips, the ports among them are not open, we still need to allow ports among them.
In order to get from one service to another on other instance on the same VPC, we need to add the port
to the firewall but in the 'Sources' section add the VPC name and delete other tags like 'All IPV4','All IPv6'.
- Now we need to add the remote services (the services on the other instance) on each docker-compose files as 'extra_hosts'
using their private ip, e.g:
extra_hosts:
- "alerts:10.136.0.2"
- "scp:10.136.0.2"
The port binding in the docker-compose is the same e.g: '- 8090:8080' just note that now the remote service must point to
the external exposed port, not the internal (like when both services are on the same instance).
[Enable Private ip]
https://www.digitalocean.com/docs/networking/vpc/how-to/enable/
[docker port binding]
In order to get from one service to another on other instance on the same VPC, we need to add the port
to the firewall but in the 'Sources' section add the VPC name and delete other tags like 'All IPV4','All IPv6'.
products/MCX/design/Esp.graphml
...@@ -83,7 +83,7 @@ ...@@ -83,7 +83,7 @@
<node id="n6"> <node id="n6">
<data key="d6"> <data key="d6">
<y:SVGNode> <y:SVGNode>
<y:Geometry height="73.88799667358398" width="61.0" x="130.97714843749998" y="545.5557533264159"/> <y:Geometry height="73.88799667358398" width="61.0" x="-31.022851562500023" y="525.331995010376"/>
<y:Fill color="#CCCCFF" transparent="false"/> <y:Fill color="#CCCCFF" transparent="false"/>
<y:BorderStyle color="#000000" type="line" width="1.0"/> <y:BorderStyle color="#000000" type="line" width="1.0"/>
<y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Dialog" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="17.96875" horizontalTextPosition="center" iconTextGap="4" modelName="custom" textColor="#000000" verticalTextPosition="bottom" visible="true" width="83.283203125" x="-11.1416015625" xml:space="preserve" y="77.88799667358398">Elasticsearch<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:ModelParameter><y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="-0.5" nodeRatioX="0.0" nodeRatioY="0.5" offsetX="0.0" offsetY="4.0" upX="0.0" upY="-1.0"/></y:ModelParameter></y:NodeLabel> <y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Dialog" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="17.96875" horizontalTextPosition="center" iconTextGap="4" modelName="custom" textColor="#000000" verticalTextPosition="bottom" visible="true" width="83.283203125" x="-11.1416015625" xml:space="preserve" y="77.88799667358398">Elasticsearch<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:ModelParameter><y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="-0.5" nodeRatioX="0.0" nodeRatioY="0.5" offsetX="0.0" offsetY="4.0" upX="0.0" upY="-1.0"/></y:ModelParameter></y:NodeLabel>
...@@ -139,6 +139,60 @@ Safety<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:Mod ...@@ -139,6 +139,60 @@ Safety<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:Mod
</y:ShapeNode> </y:ShapeNode>
</data> </data>
</node> </node>
<node id="n11">
<data key="d6">
<y:ShapeNode>
<y:Geometry height="67.0" width="93.0" x="279.78857421875" y="528.775993347168"/>
<y:Fill color="#CCFFCC" transparent="false"/>
<y:BorderStyle color="#000000" type="line" width="1.0"/>
<y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Dialog" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="17.96875" horizontalTextPosition="center" iconTextGap="4" modelName="custom" textColor="#000000" verticalTextPosition="bottom" visible="true" width="90.197265625" x="1.4013671875" xml:space="preserve" y="24.515625">Pulsar-Fluentd<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:ModelParameter><y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.0" nodeRatioX="0.0" nodeRatioY="0.0" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/></y:ModelParameter></y:NodeLabel>
<y:Shape type="roundrectangle"/>
</y:ShapeNode>
</data>
</node>
<node id="n12">
<data key="d6">
<y:ShapeNode>
<y:Geometry height="67.0" width="93.0" x="127.5771484375" y="528.775993347168"/>
<y:Fill color="#26A69A" transparent="false"/>
<y:BorderStyle color="#000000" type="line" width="1.0"/>
<y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Dialog" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="17.96875" horizontalTextPosition="center" iconTextGap="4" modelName="custom" textColor="#000000" verticalTextPosition="bottom" visible="true" width="49.15234375" x="21.923828125" xml:space="preserve" y="24.515625">Fluentd<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:ModelParameter><y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.0" nodeRatioX="0.0" nodeRatioY="0.0" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/></y:ModelParameter></y:NodeLabel>
<y:Shape type="roundrectangle"/>
</y:ShapeNode>
</data>
</node>
<node id="n13">
<data key="d4" xml:space="preserve"/>
<data key="d5"/>
<data key="d6">
<y:UMLNoteNode>
<y:Geometry height="166.77599334716797" width="499.02001953125" x="346.78857421875" y="-71.0"/>
<y:Fill hasColor="false" transparent="false"/>
<y:BorderStyle color="#000000" type="line" width="1.0"/>
<y:NodeLabel alignment="left" autoSizePolicy="content" fontFamily="Dialog" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="129.71875" horizontalTextPosition="center" iconTextGap="4" modelName="custom" textColor="#000000" verticalTextPosition="bottom" visible="true" width="466.3046875" x="16.357666015625" xml:space="preserve" y="18.528621673583984">* All Activities are published to the Pulsar under
'mcx/public/activity' topic.
* Subscriptions to 'mcx/public/activity':
1- CEP Job running under Flink
2 - Pulsar-Fluentd for forwarding activities to Elasticsearch
* Upon Alert/Service triggering from CEP a Rest Api is called from CEP
to Alerts/Scp, in the future it should be published to Pulsar as
'mcx/alerts/alert' formalerts and 'mcx/scp/service/start' or 'mcx/service/stop'
for Scp<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:ModelParameter><y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.0" nodeRatioX="0.0" nodeRatioY="0.0" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/></y:ModelParameter></y:NodeLabel>
</y:UMLNoteNode>
</data>
</node>
<node id="n14">
<data key="d6">
<y:ShapeNode>
<y:Geometry height="38.0" width="67.49800109863281" x="455.53957366943354" y="612.4873460479187"/>
<y:Fill color="#009688" transparent="false"/>
<y:BorderStyle hasColor="false" type="line" width="1.0"/>
<y:NodeLabel alignment="center" autoSizePolicy="content" fontFamily="Monospaced" fontSize="14" fontStyle="bold" hasBackgroundColor="false" hasLineColor="false" height="20.296875" horizontalTextPosition="center" iconTextGap="4" modelName="custom" textColor="#FFFFFF" verticalTextPosition="bottom" visible="true" width="29.2861328125" x="19.105934143066406" xml:space="preserve" y="8.8515625">CEP<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:ModelParameter><y:SmartNodeLabelModelParameter labelRatioX="0.0" labelRatioY="0.0" nodeRatioX="0.0" nodeRatioY="0.0" offsetX="0.0" offsetY="0.0" upX="0.0" upY="-1.0"/></y:ModelParameter></y:NodeLabel>
<y:Shape type="ellipse"/>
<y:DropShadow color="#C0C0C0" offsetX="5" offsetY="5"/>
</y:ShapeNode>
</data>
</node>
<edge id="e0" source="n0" target="n1"> <edge id="e0" source="n0" target="n1">
<data key="d10"> <data key="d10">
<y:PolyLineEdge> <y:PolyLineEdge>
...@@ -232,8 +286,19 @@ Safety<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:Mod ...@@ -232,8 +286,19 @@ Safety<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:Mod
</y:PolyLineEdge> </y:PolyLineEdge>
</data> </data>
</edge> </edge>
<edge id="e8" source="n1" target="n6"> <edge id="e8" source="n7" target="n0">
<data key="d9"/> <data key="d10">
<y:PolyLineEdge>
<y:Path sx="0.0" sy="0.0" tx="0.0" ty="0.0">
<y:Point x="161.47714843749998" y="97.27599334716797"/>
</y:Path>
<y:LineStyle color="#000000" type="line" width="1.0"/>
<y:Arrows source="none" target="standard"/>
<y:BendStyle smoothed="false"/>
</y:PolyLineEdge>
</data>
</edge>
<edge id="e9" source="n8" target="n0">
<data key="d10"> <data key="d10">
<y:PolyLineEdge> <y:PolyLineEdge>
<y:Path sx="0.0" sy="0.0" tx="0.0" ty="0.0"/> <y:Path sx="0.0" sy="0.0" tx="0.0" ty="0.0"/>
...@@ -243,21 +308,17 @@ Safety<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:Mod ...@@ -243,21 +308,17 @@ Safety<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:Mod
</y:PolyLineEdge> </y:PolyLineEdge>
</data> </data>
</edge> </edge>
<edge id="e9" source="n7" target="n0"> <edge id="e10" source="n9" target="n0">
<data key="d9"/>
<data key="d10"> <data key="d10">
<y:PolyLineEdge> <y:PolyLineEdge>
<y:Path sx="0.0" sy="0.0" tx="0.0" ty="0.0"> <y:Path sx="0.0" sy="0.0" tx="0.0" ty="0.0"/>
<y:Point x="161.47714843749998" y="97.27599334716797"/>
</y:Path>
<y:LineStyle color="#000000" type="line" width="1.0"/> <y:LineStyle color="#000000" type="line" width="1.0"/>
<y:Arrows source="none" target="standard"/> <y:Arrows source="none" target="standard"/>
<y:BendStyle smoothed="false"/> <y:BendStyle smoothed="false"/>
</y:PolyLineEdge> </y:PolyLineEdge>
</data> </data>
</edge> </edge>
<edge id="e10" source="n8" target="n0"> <edge id="e11" source="n10" target="n0">
<data key="d9"/>
<data key="d10"> <data key="d10">
<y:PolyLineEdge> <y:PolyLineEdge>
<y:Path sx="0.0" sy="0.0" tx="0.0" ty="0.0"/> <y:Path sx="0.0" sy="0.0" tx="0.0" ty="0.0"/>
...@@ -267,8 +328,28 @@ Safety<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:Mod ...@@ -267,8 +328,28 @@ Safety<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:Mod
</y:PolyLineEdge> </y:PolyLineEdge>
</data> </data>
</edge> </edge>
<edge id="e11" source="n9" target="n0"> <edge id="e12" source="n1" target="n11">
<data key="d9"/> <data key="d10">
<y:PolyLineEdge>
<y:Path sx="0.0" sy="0.0" tx="0.0" ty="0.0"/>
<y:LineStyle color="#000000" type="line" width="1.0"/>
<y:Arrows source="none" target="standard"/>
<y:EdgeLabel alignment="center" configuration="AutoFlippingLabel" distance="2.0" fontFamily="Dialog" fontSize="12" fontStyle="plain" hasBackgroundColor="false" hasLineColor="false" height="17.96875" horizontalTextPosition="center" iconTextGap="4" modelName="custom" preferredPlacement="anywhere" ratio="0.5" textColor="#000000" verticalTextPosition="bottom" visible="true" width="117.8125" x="-188.18703928186426" xml:space="preserve" y="17.280177244941797">mcx/public/activity<y:LabelModel><y:SmartEdgeLabelModel autoRotationEnabled="false" defaultAngle="0.0" defaultDistance="10.0"/></y:LabelModel><y:ModelParameter><y:SmartEdgeLabelModelParameter angle="6.283185307179586" distance="69.70217248540071" distanceToCenter="true" position="right" ratio="0.6540241221086449" segment="-1"/></y:ModelParameter><y:PreferredPlacementDescriptor angle="0.0" angleOffsetOnRightSide="0" angleReference="absolute" angleRotationOnRightSide="co" distance="-1.0" frozen="true" placement="anywhere" side="anywhere" sideReference="relative_to_edge_flow"/></y:EdgeLabel>
<y:BendStyle smoothed="false"/>
</y:PolyLineEdge>
</data>
</edge>
<edge id="e13" source="n11" target="n12">
<data key="d10">
<y:PolyLineEdge>
<y:Path sx="0.0" sy="0.0" tx="0.0" ty="0.0"/>
<y:LineStyle color="#000000" type="line" width="1.0"/>
<y:Arrows source="none" target="standard"/>
<y:BendStyle smoothed="false"/>
</y:PolyLineEdge>
</data>
</edge>
<edge id="e14" source="n12" target="n6">
<data key="d10"> <data key="d10">
<y:PolyLineEdge> <y:PolyLineEdge>
<y:Path sx="0.0" sy="0.0" tx="0.0" ty="0.0"/> <y:Path sx="0.0" sy="0.0" tx="0.0" ty="0.0"/>
...@@ -278,7 +359,7 @@ Safety<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:Mod ...@@ -278,7 +359,7 @@ Safety<y:LabelModel><y:SmartNodeLabelModel distance="4.0"/></y:LabelModel><y:Mod
</y:PolyLineEdge> </y:PolyLineEdge>
</data> </data>
</edge> </edge>
<edge id="e12" source="n10" target="n0"> <edge id="e15" source="n2" target="n14">
<data key="d9"/> <data key="d9"/>
<data key="d10"> <data key="d10">
<y:PolyLineEdge> <y:PolyLineEdge>
......
products/MCX/design/Esp.pdf
This diff could not be displayed because it is too large.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or sign in to comment