Oracle IaaS-Cloud-Native Technologies
This chapter is from the book
This chapter is from the book
This chapter is from the book
Cloud-native refers to the design and development of applications that are built specifically to take advantage of the cloud computing model. Cloud-native applications are designed to be scalable, resilient, and highly available, and they are built using microservices architecture and containerization technologies such as Docker and serverless computing.
The term cloud-native is also often used to describe applications that are designed to be deployed in a cloud environment, but it can also refer to the infrastructure and tools that support the development and deployment of these applications. This includes cloud-based platforms, tools for continuous integration and continuous delivery (CI/CD), and orchestration tools like Kubernetes. In this chapter, we will focus on several important cloud-native technologies that are commonly used by the DevSecOps team.
In general, the goal of cloud-native design is to enable organizations to build and deploy applications more quickly and efficiently, and to take advantage of the scalability, reliability, and cost benefits of the cloud.
There are several reasons why cloud-native design can be advantageous for organizations:
Scalability: Cloud-native applications are designed to be scalable, meaning they can handle a large volume of traffic or workload without experiencing performance degradation. This capability can be especially important for applications that experience sudden spikes in traffic.
Resilience: Cloud-native applications are also designed to be resilient, meaning they are able to recover quickly from failures or disruptions. This capability can help ensure that applications are available and functioning properly, even in the face of unexpected events.
Cost-Effectiveness: By leveraging the cloud, organizations can take advantage of the pay-as-you-go model to pay only for the resources they use, rather than having to invest in and maintain their own infrastructure. This model can help organizations reduce their costs and increase their efficiency by focusing on the core application logic, and not the supporting infrastructure and services.
Speed and Agility: Cloud-native design can also enable organizations to build and deploy applications more quickly and efficiently, thanks to tools like CI/CD and orchestration tools like Kubernetes. This capability can help organizations respond more quickly to changing business needs and stay competitive in the market. Part of this speed is also leveraging PaaS services like building blocks. This way, the developers can rapidly add application functionality by leveraging services managed by OCI.
Overall, the benefits of cloud-native design can help organizations build and deploy applications more effectively, take advantage of the scalability and reliability of the cloud, and reduce costs.
In this chapter, we will cover the most popular cloud-native options available on OCI, from serverless technologies like Functions, Streams, and Events, to containerized technology like Docker containers managed by Kubernetes.
Functions
Oracle Cloud Functions is a serverless computing platform offered by Oracle Cloud. It enables developers to build, deploy, and run applications and functions without having to worry about the underlying infrastructure.
With Oracle Cloud Functions, developers can write and deploy code in a variety of languages, including JavaScript, Python, Go, and Java. The platform automatically scales the code to meet demand and only charges for the actual execution time used.
Oracle Cloud Functions can be triggered by events such as a change in a database, a message being published to a message queue, or a request to an HTTP endpoint. Developers can use Oracle Cloud Functions to build and deploy a wide range of applications, including microservices, data processing pipelines, and event-driven applications.
Overall, Oracle Cloud Functions is designed to make it easier for developers to build and deploy applications quickly and efficiently, taking advantage of the scalability and cost-effectiveness of the cloud.
You can manage functions in one of three ways:
Oracle Cloud Shell: This tool enables you to use the Oracle Cloud Shell from the OCI console to create, deploy, and invoke functions. It requires minimal setup and is the easiest way to start working with functions and the OCI Command Line Interface (CLI). The shell includes OCI-specific tools and utilities that help the admin be more productive.
Local Host: This host enables you to deploy a Docker instance that enables a command-line interface to create, deploy, and invoke functions. This is a great way for macOS and Linux users to set up the CLI on their daily driver.
Oracle Compute Instance: This tool is similar to a local host but leverages some of the automation to set up Docker on a normal compute instance.
Functions can be deployed in several ways. You can use the Cloud Shell, a server, or an OCI Compute Instance. For this example, we will show how to deploy using a Linux server.
Four basic steps need to be performed:
Step 1: Setting Up the Tenancy: This step includes creating a compartment for the function to run in and the network that it will use. You also will set up the security.
Step 2: Creating the Application: Here, you will set up the application that the function will run in.
Step 3: Setting Up the Linux Host: In this step, you will get the API key and set up a Linux host to manage functions.
Step 4: Creating and Running the Function: In this last step, you will create a Python function and learn how to run it. This will also cover an example of having a function use the Python OCI API.
When using functions, you need to be familiar with a few terms:
Application: A logical grouping of functions
Function: Blocks of code stored as a Docker image
Invocation: The process of running a function
Trigger: An action that can automatically invoke a function
Context: A local installation of a development copy of a function
Provider: A service that holds the Docker image and manages the infrastructure that runs the functions
Setting Up the Tenancy
When setting up the Tenancy to run functions, you should have a security strategy that plans for what users will have access to, in order to author or run the functions. You also need to decide whether the ability to run functions is limited to a specific compartment for the application or for the entire tenancy. For this example, we will create a user group called FunctionDevelopers and include all users who can create and run functions in that group.
Next, a policy needs to be created. This policy will allow the group access to the abilities that functions require. For this example, the policy is called Functions. While the sample sets the policy for the tenancy, you can also set the allow statement for a compartment for better security. The following statements are added to the policy:
Allow group FunctionDevelopers to use cloud-shell in tenancy
Allow group FunctionDevelopers to manage repos in tenancy
Allow group FunctionDevelopers to read objectstorage-namespaces in tenancy
Allow group FunctionDevelopers to manage logging-family in tenancy
Allow group FunctionDevelopers to read metrics in tenancy
Allow group FunctionDevelopers to manage functions-family in tenancy
Allow group FunctionDevelopers to use virtual-network-family in tenancy
Allow group FunctionDevelopers to use apm-domains in tenancy
Allow group FunctionDevelopers to read vaults in tenancy
Allow group FunctionDevelopers to use keys in tenancy
Allow service faas to use apm-domains in tenancy
Allow service faas to read repos in tenancy where request.operation='ListContainer
ImageSignatures'
Allow service faas to {KEY_READ} in tenancy where request.
operation='GetKeyVersion'
Allow service faas to {KEY_VERIFY} in tenancy where request.operation='Verify'
When completed, the policy should look similar to that shown in Figure 4-1.
Creating the Application
Functions are contained within a structure known as an application. An application serves as a logical grouping of functions, providing developers with the capability to assign and set up resources for all functions within the application. Additionally, applications allow for the establishment of a shared context for storing configuration variables that can be accessed by all functions in the application while also facilitating function runtime isolation.
)
Figure 4-1 Policy for Functions
Applications are easy to create from the Console. Simply navigate to the main menu and select Developer Services > Functions > Applications. This is seen in Figure 4-2.
)
Figure 4-2 Navigating to Applications
From here, click Create Application, as shown in Figure 4-3.
)
Figure 4-3 Create Application
Next, you need to set up a few details for the function. The first is the name of the application, in this case Test Application. Next, select the VCN and subnets that the function can run in. Finally, you can select the shape. The shape is the architecture that the function will run on. You can pick Arm, X86, or both. This allows you to limit a function to a specific architecture if it requires features only available to that architecture. This step is important because the Docker image user for the function should match the architecture. For the sample, since the development is being done on Arm, Arm is selected. You can see the configuration in Figure 4-4.
)
Figure 4-4 Arm-Based Application
When done, click Create. You will then be redirected to the Application page.
Setting Up the Linux Host
To set up the function, you first need a host. The host can be nearly any modern operating system, but for this example we will use an AMD system, running Oracle Linux 8. The system will have two cores and 8 GB of RAM, with a 50 GB boot drive. The user erik was also configured to have access to root using sudo. Don’t worry about only two cores; this is more than enough for editing and testing functions.
The host setup is done in a few steps:
Step 1. Install Docker.
Step 2. Install the Fn CLI.
Step 3. Set up Fn.
Step 4. Log in to a Registry.
To install Docker, you should make sure that the addons repo is configured. This is done by running the command sudo dnf repolist, the results of which are shown in Example 4-1.
Example 4-1 Installing Docker
[erik@functions ~]$ sudo dnf repolist
repo id repo name
ol8_MySQL80 MySQL 8.0 for Oracle Linux 8 (aarch64)
ol8_MySQL80_connectors_community MySQL 8.0 Connectors Community for
Oracle Linux 8 (aarch64)
ol8_MySQL80_tools_community MySQL 8.0 Tools Community for Oracle
Linux 8 (aarch64)
ol8_UEKR7 Latest Unbreakable Enterprise Kernel
Release 7 for Oracle Linux 8 (aarch64)
ol8_addons Oracle Linux 8 Addons (aarch64)
ol8_appstream Oracle Linux 8 Application Stream
(aarch64)
ol8_baseos_latest Oracle Linux 8 BaseOS Latest (aarch64)
ol8_ksplice Ksplice for Oracle Linux 8 (aarch64)
ol8_oci_included Oracle Software for OCI users on
Oracle Linux 8 (aarch64)
[erik@functions ~]$
If ol8_addons is not installed, you can enable it by using the command sudo dnf config-manager --enable ol8_addons.
Next, you need to install Docker. If Docker is already installed, you can skip this step. To check whether Docker is installed, run the command sudo Docker version. (On most new installs, Docker is not installed.)
[erik@functions ~]$ sudo Docker version sudo: Docker: command not found [erik@functions ~]$
Next, to install Docker, you first need to add in the Extra Packages for Enterprise Linux (EPEL) library.
sudo dnf install -y epel-release
Next, point dnf to the Docker repo, using the following command:
sudo dnf config-manager -y --add-repo=https://download.Docker.com/linux/centos/ Docker-ce.repo
Next, install the community edition of Docker:
sudo dnf install Docker-ce -y
The next commands will start the server and enable it to restart on reboot:
sudo systemctl enable Docker sudo systemctl start Docker
Now, let’s make sure Docker is running by using the following command:
sudo systemctl status Docker
Check the status results to make sure Docker is running. You should see active (running) in the Active: section. Figure 4-5 shows a good example with Docker running.
)
Figure 4-5 Docker Running
You can also test that Docker is running by using it to run the hello-world container, as shown in Example 4-2.
Example 4-2 Testing That Docker Is Running by Using It to Run the hello-world Container
[erik@functions ~]$ sudo Docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
478afc919002: Pull complete
Digest: sha256:1408fec50309afee38f3535383f5b09419e6dc0925bc69891e79d84cc4cdcec6
Status: Downloaded newer image for hello-world:latest
Hello from Docker!
This message shows that your installation appears to be working correctly.
To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(arm64v8)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.
To try something more ambitious, you can run an Ubuntu container with:
$ Docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID:
https://hub.Docker.com/
For more examples and ideas, visit:
https://docs.Docker.com/get-started/
Now, add the user working on the function to the Docker group
usermod -aG Docker erik
You can test by using the command Docker version. The output should look similar to that shown in Figure 4-6.
)
Figure 4-6 Docker Version Output
On some systems, you may need to reboot for this to work for non-root users.
Next up is getting an API key, which is needed for the Fn commands to access OCI. To get an API key, from the OCI console, navigate to Profile > User Settings > Resources - API Keys. This will bring you to the API Key page shown in Figure 4-7.
)
Figure 4-7 Create a New API Key
Next, click Add API Key. This takes you to the Add API Key dialog box, where you can download the new private key, as shown in Figure 4-8.
)
Figure 4-8 API Key Dialog Box
Once the key is added, you will see the configuration file preview. Go ahead and copy the config file preview as well. Next, take the /pem file you download and the sample config and put them into the ~/.oci directory. Name the config file config.
Now edit the config file and add in the location of the .pem file. Once completed, it should look like the following.
[DEFAULT] user=ocid1.user.oc1..aaaaaaaac7z4b2mycsdmglfyflz4qomu6cmmltwq3g64vroh622cuqu4puyq fingerprint=f9:a2:62:59:df:ac:61:08:5a:87:93:98:bf:5f:61:38 tenancy=ocid1.tenancy.oc1.. aaaaaaaa257pjnvghqbiutozu4nsos4xt667ml34i4vvpyhxe2tcugfyr23e region=us-ashburn-1 key_file=/home/erik/.oci/oci.pem
Finally, secure the .pem file with the chmod command. Change the name of the file as needed.
chmod go-rwx oci.pem
Next, you need to install the Fn Project CLI by using the following command:
sudo curl -LSs https://raw.githubusercontent.com/fnproject/cli/master/install | sh
When completed, you should see a text version of the Fn logo and the version installed, as in Figure 4-9.
)
Figure 4-9 Fn Installed
Now you need to create a new context for the function, using OCI as the provider:
fn create context myappcontext --provider oracle
Next, you can use this context:
fn use context myappcontext
Now you can update the context to use the OCI credentials. In this case, use the DEFAULT profile in the Oracle config file.
fn update context oracle.profile DEFAULT
Now, set the compartment that you will use:
fn update context oracle.compartment-id \ ocid1.compartment.oc1.. aaaaaaaacqtlrn4xkj4xuyozvaljeweem7gfzxmnxe6y3j3p2hrcqjbondnq
Next, you need to point to the API URL. Each OCI region will have a unique URL. This is in the format of functions.$REGION.oci.oraclecloud.com:
fn update context api-url https://functions.us-ashburn-1.oci.oraclecloud.com
The last setup of the context is to set the object storage to be used to hold the images. The format for the Registry is $REGION.ocir.io/$NAMESPACE/$bucket:
fn update context registry iad.ocir.io/idizdwpbvdsb/function_bucket
Next up, you need an authentication token. The token is a short string of text characters; it is used to allow Docker to log in to OCI to access images. Navigate to Console > Identity > My Profile > Auth Tokens and then select the Generate Token button. You should see the dialog box shown in Figure 4-10.
)
Figure 4-10 Token Management
The next dialog box will allow you to name the token. First, pick a descriptive name and then click Generate Token, as shown in Figure 4-11.
)
Figure 4-11 Generate Token
OCI will generate a token for your use. If you lose the token, you will need to generate a new one. You are limited to only two tokens.
You can easily copy the token by clicking the Copy option in the Generated Token dialog box, shown in Figure 4-12.
)
Figure 4-12 Generated Token
Once you have the token, you can log Docker in to OCI by using the Docker login command. You will pass it the $REGION.ocir.io and the $NAMESPACE/$USER variables. The $USER should include the domain when using a user in an identity domain:
Docker login iad.ocir.io -u yournamespace/youruser
The same approach is shown in the following:
Docker login -u 'idizdwpbvdsb/Default/erik@talesfromthedatacenter.com' iad.ocir.io
When prompted for a password, use your auth token.
Now that Docker is connected, you can create a sample function. In this example, you will use the default Python HelloWorld sample. You will use the fn init command, specifying python. This will create a directory named hello in your current directory:
fn init --runtime python hello
The directory has three files:
func.py: The HelloWorld Python source code
func.yaml: The definition of the function, including the name, what runtime language to use, and also what versions are used
Requirements.txt: The libraries that need to be made available to the function when it runs
Creating and Running a Function
Next, you can deploy the application to OCI, using the fn deploy command, passing the application where the function will be located:
fn -v deploy --app "Test Application"
Now that the function is deployed, you can invoke it for testing, using the fn invoke command, passing the application and function name:
fn invoke "Test Application" hello
Now, let’s make a more complex function that will call the hello world function. You can call this function test. Use the fn init command so that you have a good start with the requirements.txt and YAML:
fn init --runtime python test
When you do this, you need to make sure that you add that to the requirements.txt file. Just add the line oci to the existing file, as shown in the following:
[erik@functions test]$ more requirements.txt fdk>=0.1.75 oci
Now, we will need some information from the existing function—mainly its OCID and invoke endpoint. You can get that from the function info page. To locate it, from the main console, navigate to the main menu and select Developer Services > Applications, then select Test Application then your application, as shown in Figure 4-13.
)
Figure 4-13 Selecting the Application
Now you should see a list of all functions for the application. Click the three dots on the right side to copy the OCID and invoke endpoint, as indicated in Figure 4-14.
)
Figure 4-14 Accessing Function Options
From here, not only can you copy information about the function, but you can also delete the function, open a support request for the function, and edit the functions. These options are available in the following pop-up function options shown in Figure 4-15.
Figure 4-15 Function Options
Save the copied endpoint and OCID. You will need it for the sample to set the correct values for the function_endpoint and function_ocid variables. This is shown in Example 4-3.
Example 4-3 Sample Function
import logging
import oci
from fdk import response
def handler(ctx, data: io.BytesIO=None):
try:
function_endpoint = "https://ew5sia72beq.us-ashburn-1.
functions.oci.oraclecloud.com/20181201/functions/ocid1.fnfunc.oc1.iad.
aaaaaaaajvhhzrlaxooujpg2tmsye4docs5mqfj6cw4lafcyqucb4izjozyq/actions/invoke"
function_ocid = "ocid1.fnfunc.oc1.iad.
aaaaaaaajvhhzrlaxooujpg2tmsye4docs5mqfj6cw4lafcyqucb4izjozyq"
function_body = ""
except (Exception) as ex:
print('ERROR: Missing key in payload', ex, flush=True)
raise
signer = oci.auth.signers.get_resource_principals_signer()
client = oci.functions.FunctionsInvokeClient(config={}, signer=signer,
service_endpoint=function_endpoint)
resp = client.invoke_function(function_id=function_ocid, invoke_function_
body=function_body)
print(resp.data.text, flush=True)
return response.Response(
ctx,
response_data=resp.data.text,
headers={"Content-Type": "application/json"}
)
You can now deploy the function:
fn -v deploy –app "Test Application"
Finally, you can invoke it as follows:
fn invoke "Test Application" test
){kind=link}