Post

[OCI] OKE Cluster 구축(2) - Network 구성

Posted
By haksuperman
views 6 min read
[OCI] OKE Cluster 구축(2) - Network 구성

해당 가이드는 아래 요건으로 구성되는 OKE Cluster 구축 가이드이며, 이외 다양한 시나리오가 가능합니다.

  1. CNI Plugin : VCN-Native Pod Networking
  2. Subnet
    1. Kubernetes API Endpoint : Private
    2. Worker Nodes : Private
    3. Pods : Private
    4. Load Balancers : Public
    5. Bastion : Public

1. 개요

1.1. 총 5개의 Subnet 구성

  1. API Endpoint용 Subnet
  2. Worker Nodes용 Sunbet
  3. Pods용 Subnet
  4. LB용 Subnet
  5. Bastion용 Subnet

1.2. 각 Subnet CIDR 대역

  1. API Endpoint용 Subnet : 10.250.0.0/29
  2. Worker Nodes용 Sunbet : 10.250.1.0/24
  3. Pods용 Subnet : 10.250.32.0/19
  4. LB용 Subnet : 10.250.2.0/24
  5. Bastion용 Subnet : 10.250.3.0/24

2. 네트워크 구성

2.1. VCN 생성

  1. Name : <적절한 VCN 이름 입력>
  2. IPv4 CIDR Blocks : 10.250.0.0/16 image

2.2. Gateway 생성

  1. Intenet gateway : <적절한 이름으로 생성> image

  2. NAT Gateway : <적절한 이름으로 생성> image

    NAT Gateway는 Public IP가 할당되며, Reserved IP를 통해 예약된 Public IP를 할당하는 것이 바람직한 구성입니다.

  3. Service Gateway : <적절한 이름으로 생성> / All ICN Services in Oracle Service Network 선택 image

2.3. Route table

2.3.1. API Endpoint Subnet용 Route table 생성

  • 0.0.0.0/0 → NAT Gateway
  • All ICN Services In Oracle Services Network → Service Gateway image

2.3.2. WorkerNode용 Route table 생성

  • 0.0.0.0/0 → NAT Gateway
  • All ICN Services In Oracle Services Network → Service Gateway image

2.3.3. Pod Subne용 Route table 생성

  • 0.0.0.0/0 → NAT Gateway
  • All ICN Services In Oracle Services Network → Service Gateway image

2.3.4. LB Subnet용 Route table 생성

  • 0.0.0.0/0 → Internet Gateway image

2.3.5. Bastion Subnet용 Route table 생성

  • 0.0.0.0/0 → Internet Gateway image

2.4. Security List

2.4.1. API Endpoint Subnet용 Security List 생성

  1. Ingress

    StateSourceProtocol/Dest. PortDescription
    Stateful10.250.1.0/24TCP/6443Kubernetes worker to Kubernetes API endpoint communication
    Stateful10.250.1.0/24TCP/12250Kubernetes worker to Kubernetes API endpoint communication
    Stateful10.250.1.0/24ICMP 3,4Path Discovery
    Stateful10.250.32.0/19TCP/6443Pod to Kubernetes API endpoint communication (when using VCN-native pod networking)
    Stateful10.250.32.0/19TCP/12250Pod to Kubernetes API endpoint communication (when using VCN-native pod networking)
    Stateful10.250.3.0/24TCP/6443External access to Kubernetes API endpoint (Bastion Subnet CIDR when access is made through OCI Bastion)

    image

  2. Egress

    StateDestinationProtocol/Dest. PortDescription
    StatefulAll ICN Services In Oracle Services NetworkTCP/ALLAllow Kubernetes API endpoint to communication with OKE
    StatefulAll ICN Services In Oracle Services NetworkICMP 3,4Path Discovery
    Stateful10.250.1.0/24TCP/10250Allow Kubernetes API endpoint to communicate with worker nodes
    Stateful10.250.32.0/19ICMP 3,4Path Discovery
    Stateful10.250.32.0/19ALL/ALLAllow Kubernetes API endpoint to communicate with pods (when using VCN-native pod networking)

    image

2.4.2. WorkerNode Subnet용 Security List 생성

  1. Ingress

    StateSourceProtocol/Dest. PortDescription
    Stateful10.250.0.0/29TCP/10250Allow Kubernetes API endpoint to communicate with worker nodes
    Stateful0.0.0.0/0ICMP 3,4Path Discovery
    Stateful10.250.3.0/24TCP/22Allow inbound SSH traffic to managed nodes
    Stateful10.250.2.0/24ALL/30000-32767Load balancer to worker nodes node ports
    Stateful10.250.2.0/24ALL/10256Allow load balancer to communicate with kube-proxy on worker nodes

    image

  2. Egress

    StateSourceProtocol/Dest. PortDescription
    Stateful10.250.32.0/19ALL/ALLAllow worker nodes to access pods
    Stateful0.0.0.0/0ICMP 3,4Path Discovery
    StatefulAll ICN Services In Oracle Services NetworkTCP/ALLAllow worker nodes to communicate with OKE
    Stateful10.250.0.0/29ALL/6443Kubernetes worker to Kubernetes API endpoint communication
    Stateful10.250.0.0/29ALL/12250Kubernetes worker to Kubernetes API endpoint communication
    Stateful0.0.0.0/0TCP/443(Custom) Accessing an external Public Image registry for kubelet

    image

(optional) Accessing an external Public Image registry for kubelet

  • 네트워크 아키텍처 상, 외부 Public Image registry 접근 불가능
  • Image Full 동작
    • Pods가 아닌 Worker nodes의 kubelet이 수행
      -> Worker nodes 서브넷의 Security list 적용 필요
    • 방화벽 Stateful Egress에만 영향 (Ingress x)
      -> 외부 Public Image registry 대역으로의 Egress 443 허용 필요 image image

2.4.3. Pod Subnet용 Security List 생성

  1. Ingress

    StateSourceProtocol/Dest. PortDescription
    Stateful10.250.1.0/24ALL/ALLAllow worker nodes to access pods
    Stateful10.250.0.0/29ALL/ALLPath Discovery
    Stateful10.250.32.0/19ALL/ALLAllow pods to communicate with other pods

    image

  2. Egress

    StateSourceProtocol/Dest. PortDescription
    Stateful10.250.32.0/29ALL/ALLAllow pods to communicate with other pods
    StatefulAll ICN Services In Oracle Services NetworkICMP 3,4Path Discovery
    StatefulAll ICN Services In Oracle Services NetworkTCP/ALLAllow pods to communicate with OCI services
    Stateful0.0.0.0/0TCP/443(optional) Allow pods to communicate with internet
    Stateful10.250.0.0/29ALL/6443Pod to Kubernetes API endpoint communication (when using VCN-native pod networking)
    Stateful10.250.0.0/29ALL/12250Pod to Kubernetes API endpoint communication (when using VCN-native pod networking)

    image

2.4.4. LB Subnet용 Security List 생성

  1. Ingress

    StateSourceProtocol/Dest. PortDescription
    Stateful0.0.0.0/0TCP/80 
    Stateful0.0.0.0/0TCP/443 

    image

  2. Egress

    StateSourceProtocol/Dest. PortDescription
    Stateful10.250.1.0/24ALL/30000-32767Load balancer to worker nodes node ports
    Stateful10.250.1.0/24ALL/10256Allow load balancer to communicate with kube-proxy on worker nodes

    image

2.4.5. Bastion Subnet용 Security List 생성

  1. Ingress

    StateSourceProtocol/Dest. PortDescription
    Stateful<Bastion 접속이 필요한 Public IP>TCP/22<Description>

    image

  2. Egress

    StateSourceProtocol/Dest. PortDescription
    Stateful0.0.0.0/0ALL/ALL 

    image

2.5. Subnet 생성

2.5.1. API Endpoint Subnet 생성

  1. Name : <적절한 이름 입력>
  2. Type : Regional
  3. CIDR Block : 10.250.0.0/29
  4. Route table : <API Endpoint Subnet용 Route table 선택>
  5. Subnet access : Private
  6. DNS Resolution : Selected
  7. DHCP Options : Default
  8. Security List : <API Endpoint Subnet용 Security list 선택> image image

2.5.2. WorkerNode Subnet 생성

  1. Name : <적절한 이름 입력>
  2. Type : Regional
  3. CIDR Block : 10.250.1.0/24
  4. Route table : <WorkerNode Subnet용 Route table 선택>
  5. Subnet access : Private
  6. DNS Resolution : 체크
  7. DHCP Options : Default
  8. Security List : <WorkerNode Subnet용 Security list 선택> image image

2.5.3. Pod Subnet 생성

  1. Name : <적절한 이름 입력>
  2. Type : Regional
  3. CIDR Block : 10.250.32.0/19
  4. Route table : <Pod Subnet용 Route table 선택>
  5. Subnet access : Private
  6. DNS Resolution : 체크
  7. DHCP Options : Default
  8. Security List : <Pod Subnet용 Security list 선택> image image

2.5.4. LB Subnet 생성

  1. Name : <적절한 이름 입력>
  2. Type : Regional
  3. CIDR Block : 10.250.2.0/24
  4. Route table : <LB Subnet용 Route table 선택>
  5. Subnet access : Private
  6. DNS Resolution : 체크
  7. DHCP Options : Default
  8. Security List : <LB Subnet용 Security list 선택> image image

2.5.5. Bastion Subnet 생성

  1. Name : <적절한 이름 입력>
  2. Type : Regional
  3. CIDR Block : 10.250.3.0/24
  4. Route table : <Bastion Subnet용 Route table 선택>
  5. Subnet access : Public
  6. DNS Resolution : 체크
  7. DHCP Options : Default
  8. Security List : <Bastion Subnet용 Security list 선택> image image

[참고] Example Network Resource Configurations

CLOUD, OCI
oci cloud oraclecloud oracle cloud infrastructure oke kubernetes
This post is licensed under CC BY 4.0 by the author.