Serve Keras Model with Flask REST API

Serve Keras Model with Flask REST API

This tutorial will briefly discuss the benefits of serving a trained Machine Learning Model with an API. Then we will take a look at a precise example using a Keras Model and Python Flask to serve the model. You’ll learn how to Serve Keras Model with Flask REST API.

Why serve ML Model with API?

Flexibility

Encapsulation the execution and manipulation of your machine learning model with an API has a few benefits. Of the benefit is the abstraction layer that you create with an (REST) API. This abstraction layer enables you to

  • test your application more easily (with tools that can send API Requests but cannot import your Tensorflow/PyTorch model directly
  • develop your application (you can initiate an execution with a REST Plugin, with your Browser or in CLI with curl
  • share functionality as a service (by deploying and making accessible via HTTPS; deploying it as a micro-service)

Mobility

Since your Model now be tweaked with pure HTTP Requests, you can deploy your Model and access/manage it via Requests. No need to login into the SSH to change a cronjob, to change a limit of SQL Query or initiate a new build to deploy a newer version.

Also, with REST API you can deploy your model easily to services like AWS ElasticBeanstalk, Google AppEngine, etc. They all need a working Server in order to deploy your app. And now you can call your Model an Application, because in fact, it is.

Serving Keras Model with Flask

The following Application Structure and Code are just one of the many possibilities how tackle this idea. If you don’t like it, you can check out the CookieCutter Template for more structure and MetaFlow for a whole complete framework.

Folder Structure

We need following folders to encapsulate the scripts, classes etc.

In the screenshot below you can see that we have a folder with different models (model_x.py). We need this separation because you could have multiple Machine Learning Models that need to be served by the same Flask Server. Optionally you can create a ML Model Loader Class that will create Machine Learning Model based a configuration file (e.g saved in yaml, json or database).

Minimal Folder Structure for a ML Model serving Application in Python.

In the queries folder you store your (in most cases very long) SQL Queries. You replace certain options with {parameter_x}, e.g: LIMIT {limit}. This way you’ll be able to dynamically generate parameterized SQL Queries reading the .sql File:

sql_file.read().format(limit=10000)

Tests/ are for tests. We are going to skip this due to scope.

Config.py is for Configurations (SQL Creds, Server Envs, etc.). We are going to skip this due to scope.

Server.py is for Flask serving our Services.

Our Services do certain actions with our model. For example: “train_service” would initiate a training process for a certain model. “prediction_service” would initiate a prediction process for a model and so on.

Machine Learning Model Class

In the code section below you can see a simple DynamicModel class with only one method which return the the compiled Keras Model. This Model does not have to be static and can be outsourced into a “build_model()” method or similar. Also, all the parameters in the layers like the input_shape should be set via the method parameters from model(). Since this is only an introductory tutorial, many useful methods are missing in this class.

from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D
from keras import backend as K
import keras

"""
Author: Andrey Bulezyuk @ German IT Academy (git-academy.com)
Date: 18.01.2020
"""

class DynamicModel():
    def __init__(self, model_name = None):
        self.model_name = model_name 

    def model(self):
        model = Sequential()
        model.add(Conv2D(32, (5, 5), input_shape=(28, 28, 1), activation='relu'))
        model.add(MaxPooling2D())
        model.add(Dropout(0.2))
        model.add(Flatten())
        model.add(Dense(128, activation='relu'))
        model.add(Dense(10, activation='softmax'))

        model.compile(loss=keras.losses.categorical_crossentropy,
                    optimizer=keras.optimizers.Adadelta(),
                    metrics=['accuracy'])
        return model
        

Service Layer

Why do we need a service layer between Flask API (server.py) and the Machine Learning Model (dynamic_model.py)? Simple. By having this extra layer (service.py) you can execute the services (in our case Class Methods) not only via REST API, but also from within other python modules.

Our service layer is responsible for importing the DynamicModel Class, loading and saving the trained model for prediction or training respectively.

import sys, os, datetime
sys.path.insert(1, os.path.join(os.getcwd(), "src/models"))
from dynamic_model import DynamicModel
from keras.datasets import mnist
from keras.utils import np_utils
from keras.models import load_model

"""
Author: Andrey Bulezyuk @ German IT Academy (https://git-academy.com)
Date: 18.01.2020
"""

class Service():

    # model_name must be supplied. 
    # otherwise no configuration cad be loaded.
    def __init__(self, model_name=None):
        self.model_name = model_name
        self.dynamic_model = DynamicModel(self.model_name)

    def _get_train_data(self):
        (x_train, y_train), (x_test, y_test) = mnist.load_data()
        # reshape to be [samples][width][height][channels]
        x_train = x_train.reshape((x_train.shape[0], 28, 28, 1)).astype('float32')
        x_test = x_test.reshape((x_test.shape[0], 28, 28, 1)).astype('float32')
        
        y_train = np_utils.to_categorical(y_train)
        y_test = np_utils.to_categorical(y_test)

        self.x_train = x_train
        self.x_test = x_test

        self.y_train = y_train
        self.y_test = y_test

    def train(self):
        # Load data
        self._get_train_data()

        # This return the compiled Keras Model from dynamic_model->model()
        print(self.y_train)
        model = self.dynamic_model.model()
        model.fit(self.x_train, self.y_train,
            batch_size=1000,
            epochs=4,
            verbose=1) 

        # Save trained model
        now = datetime.datetime.now()
        model.save(f"src/models/{self.model_name}_{now.year}{now.month}{now.day}_{now.hour}{now.minute}.h5")
        return True


    def predict(self, X):
        # Load model
        model = self._load_model()
        
        # Execute
        results = model.predict(X)
        if results is not None and results != False:
            return results
        return False

The train method works perfectly fine. You can see this in the section below when we execute it via Flask REST API with curl. The predict service method is not functional yet. The code and explanation for this is outside of the scope of this tutorial. Keep checking our IT Course Shop for similar courses with more in-depth material.

Flask API Server

Our server part is pretty simple. We import flask and our Service class. We create a route called ‘service’ with two parameters: service_name (which can be train, predict, stop, status, history, …) and model_name. Based on the parameters we execute the specified service.

import sys, os, json
sys.path.insert(1, os.getcwd())
sys.path.insert(1, os.path.join(os.getcwd(), "src"))
from flask import Flask, request
from service import Service

"""
Author: Andrey Bulezyuk @ German IT Academy (https://git-academy.com)
Date: 18.01.2020
"""

application = Flask(__name__)

@application.route("/")
def hello():
    return "Hello World!"


@application.route("/<string:service_name>/<string:model_name>", methods=["GET", "POST"])
def service(service_name=None, model_name=None):
    service = Service(model_name=model_name)

    # GET Request is enough to trigger a training process
    if service_name == 'train':
        service.train()
    # POST Request is required to get the X data for prediction process
    elif service_name == 'predict':
        service.predict()

    return f"Service: {service_name}. Model: {model_name}. Success."

if __name__ == "__main__":
    application.run(debug=True)

Example CLI & GET REquest

C:\Users\andre\code\servekeraswithapi>curl localhost:5000/train/ModelA
Service: train. Model: ModelA. Success.
Epoch 1/4
60000/60000 [==============================] - 8s 141us/step - loss: 6.5852 - accuracy: 0.7280
Epoch 2/4
60000/60000 [==============================] - 8s 140us/step - loss: 0.3276 - accuracy: 0.9141
Epoch 3/4
60000/60000 [==============================] - 8s 140us/step - loss: 0.1897 - accuracy: 0.9495
Epoch 4/4
60000/60000 [==============================] - 8s 140us/step - loss: 0.1256 - accuracy: 0.9645
127.0.0.1 - - [18/Jan/2020 20:54:53] "GET /train/ModelA HTTP/1.1" 200 -

That’s it with our short tutorial. If you liked it, subscribe to our Newsletter for more Tutorials. If you have any Questions feel free to contact us or leave a comment.

Complete Vue.js 3 Guide

Complete Vue.js 3 Guide

Last updated: 24.12.2019

We at German IT Academy are very eager to see first official release of the third Version of Vue.js 3. Therefore we are watching the developments closely and will be publishing many Posts, Tutorials & Courses on Vue.js 3. It does pay off to be on our Newsletter list (scroll down to subscribe) or check this Vue.js 3 Guide once i a while.

Also, check out the first “awesome-vue3″ repository with an updated and curated list of all resources regarding Vue.js 3.

Currently the vuejs team is working on three projects: the 2.7 Version, Maintenance of current codebase and the Roadmap (mainly the third version of Vue.js).

Major Changes in Vue.js 3

Let’s explore the official plans for third Version of Vue.js. We will also derive a great lot of content from the vue/rfcs, where all the RFCs are developed and discussed (Note: RFC = Request For Comment). We picked some of the most interesting ones and included a basic code example. As the changes become more certain, we will update our Vue.js 3 Guide to have more detailed examples.

Official RFCs December 2019

Composition API

There is a long explanation of what the Composition API is. Generally, the motivation behind it is the reuse of Logic, better Code Structure and better Type Inference. Here is a basic Example:

<template>
  <button @click="increment">
    Count is: {{ state.count }}, double is: {{ state.double }}
  </button>
</template>

<script>
import { reactive, computed } from 'vue'

export default {
  setup() {
    const state = reactive({
      count: 0,
      double: computed(() => state.count * 2)
    })

    function increment() {
      state.count++
    }

    return {
      state,
      increment
    }
  }
}
</script>

New syntax for scoped slots usage

  • New directive v-slot that unifies slot and slot-scope in a single directive syntax.
  • Shorthand for v-slot that can potentially unify the usage of both scoped and normal slots.
<!-- default slot -->
<foo v-slot="{ msg }">
  {{ msg }}
</foo>

<!-- named slot -->
<foo>
  <template v-slot:one="{ msg }">
    {{ msg }}
  </template>
</foo>

Shorthand syntax for the v-slot syntax

<foo>
  <template #header="{ msg }">
    Message from header: {{ msg }}
  </template>

   <template #footer>
    A static footer
  </template>
</foo>

Dynamic values in directive arguments

<div v-bind:[key]="value"></div>
<div v-on:[event]="handler"></div>

# instead of

<div v-bind="{ [key]: value }"></div>
<div v-on="{ [event]: handler }"></div>

Global API

Currently in 2.x, all global APIs are exposed on the single Vue object:

import Vue from 'vue'

Vue.nextTick(() => {})

const obj = Vue.observable({})

In 3.x, they can only be accessed as named imports:

import Vue, { nextTick, observable } from 'vue'

Vue.nextTick // undefined

nextTick(() => {})

const obj = observable({})

Re-design app bootstrapping and global API

Global APIs that globally mutate Vue’s behavior are now moved to app instances created the new createApp method, and their effects are now scoped to that app instance only.

// Before
import Vue from 'vue'
import App from './App.vue'

Vue.config.ignoredElements = [/^app-/]
Vue.use(/* ... */)
Vue.mixin(/* ... */)
Vue.component(/* ... */)
Vue.directive(/* ... */)

new Vue({
  render: h => h(App)
}).$mount('#app')


// After
import { createApp } from 'vue'
import App from './App.vue'

const app = createApp()

app.config.ignoredElements = [/^app-/]
app.use(/* ... */)
app.mixin(/* ... */)
app.component(/* ... */)
app.directive(/* ... */)

app.mount(App, '#app')

More Material on Vue.js 3

  • Github Repository ‘awesome-vue3‘ with a compilation of most recent and most popular Vue.js 3 Resources.
IT Career Path Nobody Is Discussing

IT Career Path Nobody Is Discussing

Whatever path you decide on, we will help you find out the ideal things to do to take. Your career path contains the jobs you should hit your final career objective. While the career path will appear different for everybody, aiming to acquire equal parts experience in company and technology is your safest bet. For that reason, it’s essential you select the ideal career path to ensure it’s authentic to who you are.

Nuance of IT Career Path

For midmarket CIOs, it’s an incredibly exciting prospect for them to not only grow their careers but also to make a great deal of money,” Banerji explained. Information Technology (IT) careers are predicted to experience significant growth during the present decade. When you’re ready to take that very first step into your new IT career you will begin with the fundamentals.

Roadmap for IT Career Path

One particular tangible way to begin your career change is by way of freelance or part-time work. As soon as you have decided that a career change is certainly the way forward, there’s still a lot you will need to think about. Making a career change is just one of the toughest job-search challenges. Career change, as soon as you have decided that a career change is certainly the way forward, there’s still a lot you will need to think about. Whether the career change is voluntary or involuntary, individuals may experience a number of emotions like fear, anxiety, or a feeling of loss. Tons of individuals wonder whether making a significant career change is well worth it.

The Foolproof IT Career Path Strategy

Figure out if career change is appropriate for you and how to change careers. Irrespective of where you currently are in your career, there are a couple of things you can begin doing to increase your odds of a thriving future IT career path. Another way to go into a career in information technology is to make a college degree. Online Courses are a quicker way though. In addition you get the most up to date skills.

You are able to still receive the job. You can proceed and apply for jobs that you believe you’d enjoy, whether you’ve got the requirements listed in the work ad or not. Simply perusing books about the market, in addition to specific topics like programming and networking, can help you explore the range of work in the area.

Growing & Goal

Much like any goal, it can help to define where you wish to go and a path to get there. It’s possible to segue from just about any career path into any other. You’re able to pursue a few different, new career paths simultaneously.

As you grow, however, the path forward can come to be not as clear. A career path provides the employee a feeling of direction, a way to evaluate career progress, and career targets and milestones. Before you go out and select a career path, it’s imperative that you take the opportunity to figure out what success means to you. Whichever way a career path takes someone, it’s intended to provide greater satisfaction of a worker’s career values and requires by targeting a string of jobs created to receive them to their career objective. Other career paths are indirect and could involve work in various industries or kinds of jobs, including when someone is working on a career change.

Awesome Future of Frontend in 2020

Awesome Future of Frontend in 2020

Before we dive in to Future of Frontend in 2020. Let’s begin with some basics as we gradually get more forward-looking. We obviously cannot know what technology will dominate in 2020 and what framework will pop up. But we can try at least.

What is frontend?

Frontend is referred to as the pattern of transforming data to a graphical interface with the use of CSS, HTML, and JavaScript, so that users can easily view and efficiently interact with the data. It distinctively translates a computer programming source code into an intermediate illustration to produce code in a computer output language.

It is also a common term used by programmers and computer programmers to describe part of the layers that make up a website, computer program or hardware, which are depicted based on how accessible they are to a user. The frontend layer strategically placed on top of the backend includes all software or hardware that is part of a user interface.

Professionals such as web designers usually handle Frontend section of every project and the components are customer facing. Some of these components includes:

  • Search engine optimization (SEO)
  • Graphic design and image editing tools
  • Design and markup languages such as CSS and HTML
  • Web performance and browser compatibility
  • Usability and accessibility testing

Frontend Career

Frontend career is an interesting area with availability of various great remunerations as a profession that becomes easier with the adequate knowledge of the three primary coding languages i.e. CSS, HTML and JavaScript. Research has deducted that frontend development is and will always be a good career path for humans because software keeps evolving every day.

The consistent demands for frontend career specialist is very high. Some of the key skills of a frontend developer are responsive design, frameworks, debugging, web performance, CSS preprocessing and command line among others.

Consider taking our VueJS Certification or VueJS Online Course. It’s on-demand, easy to understand and has tremendous benefits for you career.

Relevant Areas of Frontend

An important area where frontend (client-side programming) is of high relevance is UI, UX. There is a distinct relationship between frontend, UI and UX. They work together seamlessly. The UX design centered on the satisfaction of the user experiences with software. Frontend development is the technical implementation of the software user interface.

UI design is the graphical bridge that connects the UX and frontend. An individual can choose to be a UI, UX, and frontend web developer in which he/she will be responsible for applying interactive and visual design experience. For you to be successful in this field, you must be able to observe users behavior to bring the best out of an application. Your primary aim should be to ensure user-based company goals are been reached satisfactorily. Some of the major importance of frontend development in relationship with UI and UX are:

  • Optimize navigation: Intuitive navigation will help gain customer trust by ensuring that the visitors find whatever they are seeking from your site. It majorly comprises of a well-planned site layout, clean, and structured impressive graphics.
  • Visitor retention: This will help increase traffic and conversion. Thus, optimized performance is one of the business benefits of front-end development.

Some other sub areas of its relevance are mobile frontend and web frontend:

  • Mobile Frontend: This can operate effectively without an active internet.
  • Web Frontend: This requires active internet for it work properly on your devices.

Generally, frontend development tools are focused on the user interface and user experiences. These has given birth to the following importance:

  • Creating modern day responsive websites or web app for mobility segment
  • Building bug free, secured and consistent products for high traffic web zone
  • Developing quickly reacting features or interactive app tools for online stores
  • Easy to learn, use and scale technologies, etc

In conclusion, the earlier you learn various frontend skills such as VueJS today – the better. Frontend courses are made easy with the best-qualified teaching procedures at German IT Academy.

Frontend in 2020

As for the prediction of where Frontend will head in 2020 it’s no easy task. Just as VueJS came out of nowhere and allowed more people with lower frontend skills to participate in the development. So can another framework, paradigm or tools do so in 2020.

Responsive stays

It is certainly though that Material Design & Bootstrap are going to continue growing like they already do. They will keep evolving and doing the little hard jobs of micro-designing small buttons and tables for us.

Changing Screen Sizes

If the rise of foldable phones continues, the frontend developers will need to adjust to this new environment of suddenly changing screen size and making sure that the app, page, game works transitions seamlessly between both screen sizes.

Performance & Data Focus

With the rise of data lakes and data on itself, Frontend Industry will be coming up with ways to make their Application lighter and more intelligent in showing, grouping and filtering the right information without overloading the hardware, browser, etc.

Artificial Intelligence in Frontend

With AI supported in-app & on website behaviour analysis of end consumers. We will have insight into deep psychology of our brains. What color triggers more excitement, what button animation triggers more dopamine, etc. The Frontend world will definitely become more fine tuned to our psychological “needs” in our to manipulate the end consumer to … consume.

Frontend in VR & AR

It’s totally new field for us. But one can imagine that Frontend will play a crucial role in AR and VR. This area brings a whole new set of challenges with itself. The user does not use a mouse or a keyboard. The user does stare a small screen. There could pop up a Framework like “ReactVR” (e.g. React 360) or something like that. Which would allow you do design 360 User Interfaces. Or something like “VueAR”, which would allow you to create transparent overlay User Interfaces to allow the user to use your app while they are interacting with their environment.

How to Run Moodle with Docker Compose

How to Run Moodle with Docker Compose

In this tutorial we are going to have a Moodle Instance up and running on a Virtual Machine – inside Docker Containers.

First of all, install and get Docker up and running. Then, download Moodle or make sure you have a Git-Repository ready at hand. We are going to use docker-compose for this task. Now let’s try and Run Moodle with Docker.

Moodle with Docker

We need a container that will have all the needed technology to run and serve the Moodle source code (your repository). This configuration will come from an Image, that we will use and configure through a docker-compose.yml file. Create a file docker-compose.yml and insert the version of docker-compose syntax that we are going to use:

version: "3.3"

These Technologies are needed:

  • Moodle Container
    • Operating System (Ubuntu)
    • Server (Apache2, Nginx)
    • PHP7.x (incl. php-curl, php-mysql, etc.)
  • Database Container
    • MariaDB 10+

Database Docker Image

We are going to use a simple and straightforward solution from bitnami. We simple create a service that will use this image, and set some parameters like username, db_name, etc.

services:
  mariadb:
    image: "bitnami/mariadb:10.1"
    networks:
      - moodle-net
    environment:
      - MARIADB_USER=bn_moodle
      - MARIADB_DATABASE=bitnami_moodle
      - ALLOW_EMPTY_PASSWORD=yes
    volumes:
      - type: bind
        source: /home/maria_data
        target: /bitnami
    ports:
      - "3306:3306"

We call our service ‘mariadb’, expose some ports (3306) and bind a volume to persist our data on our host machine. We also attach ‘mariadb’ to a network moodle-net. Also, make sure that docker can read/write the /home/maria_data folder on your host. We set rwx access for now (which is not recommended), please make sure to set the proper rights.

sudo chmod a+rwx /home/maria_data/

Server Docker Image

There are already built images for these technologies: webdevops/php-apache:7.3 & bitnami/mariadb.

Let’s pull and test the PHP & Apache image first.

#Download
docker pull webdevops/php-apache:7.3

# Run Image (Copy image ID. 'docker images')
docker run -d -p 80:80 efec3d223189

Now navigate to localhost or your public IP Address. You should see an empty Apache2 page.

Moodle and Docker

This page is the result of the file ‘index.php’ in container in folder /app. Worked fine. To use this image as a service in our config file, we need to create a new service in docker-compose.yml.

  moodle:
    image: "webdevops/php-apache:7.3"
    networks:
      - moodle-net
    environment:
      - MOODLE_DBHOST=mariadb
      - MOODLE_DBPORT=3306
      - MOODLE_DBUSER=bn_moodle
      - MOODLE_DBPASS=''
      - MOODLE_DBNAME=bitnami_moodle
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - type: bind
        source: /home/moodle_data
        target: /moodledata/moodledata
      - type: bind
        source: /home/gita-moodle
        target: /app
    depends_on:
      - mariadb

We specify the image that we tested previously. Specify the network to be same as the network of mariadb container. The Env Vars that we define, will be used by moodle in config.php. We open ports and specify the volumes. We make sure that moodle container only starts after mariadb container is up with the depends_on option. Make sure /home/moodle_data is writable.

sudo chmod a+rwx /home/moodle_data/

Before we continue, in these courses you can learn how to create backend and frontend application. Both of which you can easily deploy and serve with Docker. And now you know how to use docker-compose. Expand your skill-set with these courses.

Finish line

At last, we need to define our network and volumes:

networks:
  moodle-net:
    driver: bridge

volumes:
  mariadb_data:
    driver: local
  moodle_data:
    driver: local

Now let’s boot up these containers with and visit localhost or your public ip.

docker-compose up -d

You should see the installation process. Enter your Database data (see ENV VARS in MariaDB Service. After that a config.php file will be created. If not, copy the code and create a config.php file in root folder.

Moodle Installation process Docker

After finishing up the setup process of moodle, you’ll finally see your ready working moodle instance. Now you run Moodle with Docker Containers.

After setting up moodle. Your host volumes/folders will be filled with data that comes from containers. E.g. moodle population a predefined folder structure in moodle_data.

ls moodle_data/
cache  filedir  lang  localcache  lock  moodle  muc  sessions  temp  trashdir

Same, and more importantly, applies to mariadb container. Our Database is now persisted on our host machine.

/home/maria_data/mariadb/data# pwd
/home/maria_data/mariadb/data
/home/maria_data/mariadb/data# ls
aria_log.00000001  aria_log_control  bitnami_moodle  ibdata1  ib_logfile0  ib_logfile1  multi-master.info  mysql  mysql_upgrade_info  performance_schema  tc.log  test

Andrey Bulezyuk

German IT Academy Coach

Andrey Bulezyuk is the founder of German IT Academy and a course creator. He mainly covers topics from Web Development (NodeJS, VueJS, Django etc.) to Data Science (Pandas, Numpy) and Machine Learning (Tensorflow, Keras, etc.). He published a German book “Algorithmic Trading“, giving his readers the opportunity to learn how to code automatic trading systems for the stock market or the Forex. He has experience as a Backend and Frontend Developer. In his early years he worked on Web Projects as a Freelancer. Switched later to Financial Market System Developing. Currently employed as Machine Learning Engineer. He takes part in different projects and acts as a Co-Founder in different small start-ups. Andrey Bulezyuk was an employed Full-Stack Developer in Munich. Currently his duties do cover areas of Machine Learning and ETL.