Jenkins: Make an agent connection to EC2 instance.

Conditions

  • EC2 instance has no Elastic IP.
  • When an agent has been in idle status for some times, that instance has to be stopped.

Script

#!/bin/bash
set -x
export PATH=/vol/users/gatekeeper.tvsw/.local/bin:$PATH
ID=${1}
echo "INFO: Jenkins Url = ${JENKINS_URL}" >&2
echo "INFO: Start EC2 instance - ${ID}" >&2
aws ec2 start-instances --instance-ids ${ID} >&2

echo "INFO: Wait until EC2 ${ID} will be 'Running'" >&2
aws ec2 wait instance-running --instance-ids ${ID} >&2

echo "INFO: Wait for 10 seconds" >&2
sleep 10
echo "INFO: Get ${ID}s PublicIpAddress" >&2
IP=`aws ec2 describe-instances --instance-ids ${ID}|grep PublicIpAddress |sed 's/\ //g'|sed 's/\"//g'|sed 's/\,//g'|awk -F: '{print $2}'`
NETWORKIFID=`aws ec2 describe-instances --instance-ids ${ID}|grep NetworkInterfaceId|sed 's/\ //g'|sed 's/\"//g'|sed 's/\,//g'|awk -F: '{print $2}'`
echo "INFO: ${ID}s PublicIpAddress = ${IP}" >&2
echo "INFO: ${ID}s NetworkInterfaceId= ${NETWORKIFID}" >&2

trap term_action SIGTERM
function term_action() {
echo "INFO: Stop an instance: ${ID}" >&2
aws ec2 stop-instances --instance-ids ${ID} 2>/dev/null
>&2 echo "INFO: Stopped:${ID}"
}

date >&2
rm -rf slave.jar
wget --auth-no-challenge --no-check-certificate ${JENKINS_URL}/jnlpJars/slave.jar
scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null slave.jar ubuntu@${IP}:~/slave.jar >&2
ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -n ubuntu@${IP} 'ls -al ~/slave.jar' >&2
ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -n ubuntu@${IP} mk_disk_i3 >&2
ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -n ubuntu@${IP} mkdir -p ${WORKSPACE} >&2
ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -n ubuntu@${IP} 'echo Postech2001! | sudo -b openconnect --juniper svlvpn.lge.com -u allessunjoo.park --passwd-on-stdin >> ~/.openconnect_watchdog.log 2>&1 &'
ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null ubuntu@${IP} 'java -jar ~/slave.jar'
term_action
set +x

Configuration on Jenkins

Advertisements

Make IAM user to only start and stop a designated EC2 instance

Prerequisites

Steps

  1. Go to ‘IAM’ and click ‘Policies’ on the left menu. Click ‘Create policy’ button to make a custom policy
    policy-menu
  2. Select one of policy tools. I will select ‘Create “Your Own Policy’ for a example.
    create-policy

      • Required Actions
        • ec2:StartInstances
          • Target Resource: EC2 instance’s arn name
        • ec2:StopInstances
          • Target Resource: EC2 instance’s arn name
        • ec2:Describe*
          • Target Resource: All EC2 instances = “*”
      • Example for a policy document
        {
            "Version": "2012-10-17",
                "Statement": [
                {
                "Sid": "Stmt1501742713000",
                "Effect": "Allow",
                "Action": [
                   "ec2:StartInstances",
                   "ec2:StopInstances"
                ],
                "Resource": [
                   "arn:aws:ec2:ap-northeast-2:652050604906:instance/i-034dd11af16acf000"
                ]
                },
                {
                "Effect": "Allow",
                "Action": "ec2:Describe*",
                "Resource": [
                   "*"
                ]
                }
             ]
         }
  3. Go to https://console.aws.amazon.com/iam/home#/users and select a user
    select-iam-user
  4. Select ‘Permissions’ tab and click ‘Add Permissions’
    add-permission
  5. Click ‘Attach existing policies directly’ and select a policy that you created. Click ‘Next: Review’ and ‘Add permissions’ if nothing is problem.
    ad-permissions-select-policy.png

Jenkins Github Plugin Configurations

  1. Install GitHub Plugin
  2. Go to https://github.com/settings/tokens and Click ‘Generate new token’
    Screen Shot 2017-07-21 at 2.54.40 PM
    Screen Shot 2017-07-21 at 2.55.40 PM.png
  3. On Jenkins Site, Go to ‘Manage Jenkins’ > ‘Configure System’ > ‘GitHub section
  4. Select ‘Add GitHub Server > GitHub Server’
    Screen Shot 2017-07-21 at 2.59.55 PM.png
  5. Click ‘Credentials > Add Jenkins’ and Write a generated token from GitHub with additional information
    • Domain: Global credentials
    • Kind: Secret text
    • Scope: Global
    • Secret: Generated token value
    • ID: Distinct text for this credential
      Screen Shot 2017-07-21 at 3.02.06 PM.png
  6. Select a credential you created and Click ‘Test Connection’
    Screen Shot 2017-07-21 at 3.13.35 PM.png
  7. Go to a build job configuration and Change configuration
    • Source Code Management
      Screen Shot 2017-07-21 at 3.35.37 PM.png
    • Build Trigger
      Screen Shot 2017-07-21 at 3.35.51 PM.png

Allow that only instance A can connect to instance B and Prevent the others on a public internet trying to connect to instance B

Case 1: Only instance A can connect to instance B and instance B can’t initiate a connection to a public internet.

Conditions

  • Instance A can initiate a connection to a public internet
  • A public internet can initiate a connection to instance A with its public IP address
  • Instance A can make a connection to instance B with private IP address
  • Instance B has no public IP address
  • A public internet can’t initiate a connection with instance B

How to define

  1. Create your VPC
    VPC Name: my_vpc
  2. Create an internet gateway  and attach it to VPC
    • Create an internet gateway: ig_for_my_vpc
    • Attach ‘ig_for_my_vpc‘ to ‘my_vpc‘: Click ‘Attach to VPC
    •  

      This slideshow requires JavaScript.

  3. Create a subnet ‘subnet_external’
    • In ‘VPC’ box, select a VPC that you created
    • IPv4 CIDR block:
      • Declare that this subnet use IP prefix 156.147.1.0,
      • ‘IPv4 CIDR block:’ 156.147.1.0/24
    • Screen Shot 2017-07-17 at 5.50.08 PM
    • Add a routing rule to VPC’s internet gateway on ‘subnet_external’s routing table
    • subnet_external_rt
      Click a routing table text

      subnet_external_rt_edit
      Write a routing rule to an internet gateway
  4. Create a routing table for internal network
    • Routing table name: rt_subnet_internal
      Select VPC that you created.
      rt_create
  5. Create a subnet ‘subnet_internal’
    • In ‘VPC’ box, select a VPC that you created
    • IPv4 CIDR block:
      • Declare that this subnet use IP prefix 156.147.2.0,
      • ‘IPv4 CIDR block:’ 156.147.2.0/24

        subnet_internal
        Write information for ‘subnet_internal’
  6. Create an EC2 instance with ‘subnet_external’
    ec2_external
    EC2 external instance configuration

    ec2_external_status
    EC2-external: Public IP = 13.124.183.176, Private IP = 156.147.1.228
  7. Create an EC2 instance with ‘subnet_internal’
    • You have to select ‘Disabled‘ for ‘Auto-assign Public IP
      ec2_internal.png
      EC2 configuration

      ec2_internal_status
      EC2-external: Private IP = 156.147.2.159
  8. Try to check if you can connect to EC2 internal instance after logging onto EC2 external instance.


Sunjoo:~ sunjoo$ ssh -i key.pem ubuntu@13.124.183.176
Warning: Permanently added '13.124.183.176' (ECDSA) to the list of known hosts.
ubuntu@ip-156-147-1-228:~$ hostname
ip-156-147-1-228
ubuntu@ip-156-147-1-228:~$ ssh ubuntu@156.147.2.159
Warning: Permanently added '156.147.2.159' (ECDSA) to the list of known hosts.
ubuntu@ip-156-147-2-159:~$ hostname
ip-156-147-2-159
ubuntu@ip-156-147-2-159:~$

Use ldap-based authentication per a context

This article will be useful if you want to use a separated authentication system for each context with Apache Tomcat

Why?

Apache Tomcat’s manager application has its own security roles. If you define  configuration in server.xml ( after deleting “),  a Manager application deployed in a same host can’t use its own roles/users configuration.

If you can create roles with same names in a LDAP server  and add users to it, that’s a simple solution. But you have to request it to LDAP server manager if you don’t have rights.

This configuration help you to deploy an web application without requesting LDAP server configuration changes..

How to

  1. Create a META-INF/context.xml in a context’s application directory with default contents
    <?xml version="1.0" encoding="UTF-8"?>
    <Context antiResourceLocking="false" privileged="true" >
    
    </Context>
    
  2. Create  definition in  tag
    <Context antiResourceLocking="false" privileged="true" >
    <Realm className="org.apache.catalina.realm.JNDIRealm" debug="99" connectionURL="ldap://192.168.0.12:3268" authentication="simple" referrals="follow" connectionName="CN=ad_user,CN=Users,DC=Apache,DC=NET" connectionPassword="xxxxxx" userSearch="(sAMAccountName={0})" userBase="dc=Apache,dc=NET" userSubtree="true" roleSearch="(cn={0})" roleName="cn" roleSubtree="true" roleBase="dc=Apache,dc=NET">
    </Realm>
    </Context>
    
  3. Add this xml code in web.xml
    <web-app>
       <!-- Other codes -->
        <security-constraint>
            <web-resource-collection>
                <web-resource-name>opengrok.org</web-resource-name>
                <description></description>
                <url-pattern>/*</url-pattern>
            </web-resource-collection>
            <auth-constraint>
                <role-name>*</role-name>
            </auth-constraint>
        </security-constraint>
        <login-config>
            <auth-method>FORM</auth-method>
            <form-login-config>
                <form-login-page>/login.html</form-login-page>
            </form-login-config>
        </login-config>
        <security-role>
            <role-name>*</role-name>
        </security-role>
    </web-app>
    

    If you change <auth-method>‘s value to ‘BASIC’, you can remove <form-login-config> tag and use Browser’s default login dialog.

  4. Create login.html page
    <form action="j_security_check" method="POST">
    <table>
    <tbody>
    <tr>
    <td colspan="2">Login to the TVL's OpenGrok System</td>
    </tr>
    <tr>
    <td colspan="2">Use LGE AD account and password:</td>
    </tr>
    <tr>
    <td>Name:</td>
    <td><input name="j_username" type="text" /></td>
    </tr>
    <tr>
    <td>Password:</td>
    <td><input name="j_password" type="password" /></td>
    </tr>
    <tr>
    <td colspan="2"><input type="submit" value="Go" /></td>
    </tr>
    </tbody>
    </table>
    </form>
    
    
  5. Restart Tomcat or a context itself.