Instant Microservices: Rules for Logic and Security

1. Instant Microservices: Rules for Logic and Security

See how to build a complete database system with one command: a multi-page web app and a multi-page API. Customize with rules and Python.

In this article, see how to build a complete database system, in minutes instead of weeks or months:

1.  An API, and, we'll add UI and logic to make it a microservice

2.  Logic and security: Multi-table constraints and derivations, and role-based security

3.  An Admin app: And finally, a multi-page, multi-table web app

We'll use API Logic Server (open source), providing:

The entire process takes 10 minutes, instead of several weeks using traditional development.

You can use this article in several ways:

• Conceptual overview: The main article focuses on the concepts and basic process. Operational details are moved to the Appendix to retain focus.
• Self-demo: Install and create this system yourself.

1. Automation: Instant Project

This project was created with a command like:

$ ApiLogicServer create --project_name=basic_demo --db_url=basic_demo

Note: the db_url value is an abbreviation.  For your own databases, you would supply a SQLAlchemy URI.

This creates a project by reading your schema. The database is Customer, Orders, Items, and Product, as shown in the Appendix. 

You can open it with VSCode, and run it as follows:

1. Create a virtual environment: As shown in the Appendix

2. Start the server: F5 (also described in the Appendix)

3. Start the Admin App: Either use the links provided in the IDE console or click http://localhost:5656/. The screen shown below should appear in your Browser.

The sections below explore the system that has been created (which would be similar for your own database).

API With Swagger

The system creates an API with endpoints for each table, with filtering, sorting, pagination, optimistic locking, and related data access - self-serve, ready for custom app dev.

Admin App

It also creates an Admin App: multi-page, multi-table- ready for business user agile collaboration, and back office data maintenance. This complements custom UIs created with the API.

You can click Customer 2, and see their Orders, and Items.

2. Customize in Your IDE

While API/UI automation is a great start, it's critical to enforce logic and security. Here's how.

The following apply_customizations process simulates adding security to your project, and using your IDE to declare logic and security in logic/declare_logic.sh and security/declare_security.py. Declared security and logic are shown in the screenshots below.

To apply customizations, in a terminal window for your project:

1.     Stop the server (red stop button, or Shift-F5 - see Appendix)

2.     Apply Customizations

# mac, linux

sh apply_customizations.sh

#windows

./apply_customizations.ps1

Declare Security

The apply_customizations process above has simulated the ApiLogicServer add-authcommand, and using your IDE to declare security in logic/declare_logic.sh.

To see security in action:

1.  Start the server F5

2.  Start the Admin app: http://localhost:5656/

3.  Login as s1, password p

4.  Click Customers

Observe:

1.  The admin app now shows fewer customers

2.  The screenshot below illustrates the security declaration and operation:

o    The declarative Grants in the upper code panel, and

o    The logging in the lower panel, to assist in debugging by showing which Grants (+ Grant:) are applied:

Declare Logic

Logic (multi-table derivations and constraints) is a significant portion of a system, typically nearly half. API Logic server provides spreadsheet-like rules that dramatically simplify and accelerate logic development.

Rules are declared in Python and simplified with IDE code completion. The screen below shows the 5 rules for Check Credit Logic.

The apply_customizations process above has simulated the process of using your IDE to declare logic in logic/declare_logic.sh.

To see the logic in action:

1.     In the admin app, Logout (upper right), and log in as admin, p.

2.     Use the admin app to add an Order and Item for Customer 1 (see Appendix).

Observe the rules firing in the console log, as shown in the next screenshot.

Logic provides significant improvements over procedural logic, as described below.

A. Complexity Scaling

The screenshot below shows our logic declarations and the logging for inserting an Item. Each line represents a rule firing and shows the complete state of the row.

Note that it's a Multi-Table Transaction, as indicated by the indentation. This is because - like a spreadsheet - rules automatically chain, including across tables.

B. 40X More Concise

The 5 spreadsheet-like rules represent the same logic as 200 lines of code, shown here. That's a remarkable 40X decrease in the backend half of the system.

C. Automatic Re-Use

The logic above, perhaps conceived for Place order, applies automatically to all transactions: deleting an order, changing items, moving an order to a new customer, etc. This reduces code and promotes quality (no missed corner cases).

D. Automatic Optimizations

SQL overhead is minimized by pruning, and by elimination of expensive aggregate queries. These can result in orders of magnitude impact.

E. Transparent

Rules are an executable design. Note that they map exactly to our natural language design (shown in comments) - readable by business users. 

Optionally, you can use the Behave TDD approach to define tests, and the Rules Report will show the rules that execute for each test.

3. Iterate With Rules and Python

Not only are spreadsheet-like rules 40X more concise, but they meaningfully simplify maintenance. Let's take an example:

Give a 10% discount for carbon-neutral products for 10 items or more.

The following apply_iteration process simulates an iteration:

• Acquires a new database with Product.CarbonNeutral
• Issues the ApiLogicServer rebuild-from-database command that rebuilds your project (the database models, the API), while preserving the customizations we made above
• Acquires a revised ui/admin/admin.yaml that shows this new column in the admin app
• Acquires this revised logic: In logic/declare_logic.py, we replaced the 2 lines for the models.Item.Amount formula with this (next screenshot shows revised logic executing with breakpoint):

    def derive_amount(row: models.Item, old_row: models.Item, logic_row: LogicRow):

        amount = row.Quantity * row.UnitPrice

        if row.Product.CarbonNeutral and row.Quantity >= 10:

           amount = amount * Decimal(0.9)  # breakpoint here

        return amount

    Rule.formula(derive=models.Item.Amount, calling=derive_amount)

To apply this iteration, in a terminal window for your project:

1.     Stop the server (red stop button, or Shift-F5 - see Appendix)

2.     Apply iteration

# mac, linux

sh apply_iteration.sh

#windows

./apply_iteration.ps1

3. Set the breakpoint as shown in the screenshot below

4. Test: Start the server, log in as Admin

5. Use the Admin App to update your Order by adding 12 Green items

At the breakpoint, observe you can use standard debugger services to debug your logic (examine Item attributes, step, etc).

This simple example illustrates some significant aspects of iteration, described in the sub-sections below.

A. Maintenance Automation

Along with perhaps documentation, one of the tasks programmers most loathe is maintenance. That's because it's not about writing code, but it's mainly archaeology - deciphering code someone else wrote, just so you can add 4 or 5 lines that will hopefully be called and function correctly.

Rules change that since they self-order their execution (and pruning) based on system-discovered dependencies. So, to alter logic, you just "drop a new rule in the bucket," and the system will ensure it's called in the proper order, and re-used over all the use cases to which it applies. Maintenance is faster, and higher quality.

Extensible With Python

In this case, we needed to do some if/else testing, and it was convenient to add a pinch of Python. Using "Python as a 4GL" is remarkably simple, even if you are new to Python.

Of course, you have the full object-oriented power of Python and its many libraries, so there are no automation penalty restrictions. 

Debugging: IDE, Logging

The screenshot above illustrates that debugging logic is what you'd expect: use your IDE's debugger. This "standard-based" approach applies to other development activities, such as source code management, and container-based deployment.

Customizations Retained

Note we rebuilt the project from our altered database, illustrating we can iterate while preserving customizations.

4. API Customization: Standard

Of course, we all know that all businesses the world over depend on the hello world app. This is provided in api/customize_api. Observe that it's:

• Standard Python
• Using Flask
• For database access, SQLAlchemy: Note all updates from custom APIs also enforce your logic.

5. Deploy Containers: Collaborate

API Logic Server also creates scripts for deployment. While these are not required at this demo, this means you can enable collaboration with business users:

1.     Create a container from your project - see devops/docker-image/build_image.sh

2.     Upload to Docker Hub

3.     Deploy for Agile collaboration 

Summary

In minutes - not days or weeks - you've used API Logic Server to convert an idea into working software, customized logic and security, and iterated to meet new requirements.

To dive deeper, you can install API Logic Server and execute this demo - or create a system from your own databases.

Appendix: Database Schema

Appendix: Procedures

Specific procedures for running the demo are here, so they do not interrupt the conceptual discussion above.

You can use either VSCode or Pycharm.

1. Establish Your Virtual Environment

Python employs a virtual environment for project-specific dependencies. Create one as shown below, depending on your IDE.

For VSCode:

Establish your venv, and run it via the first pre-built Run Configuration. To establish your venv:

python -m venv venv; venv\Scripts\activate     # win

python3 -m venv venv; . venv/bin/activate      # mac/linux

 pip install -r requirements.txt

For PyCharm, you will get a dialog requesting to create the venv; say yes.

See here for more information.

2. Start and Stop the Server

Both IDEs provide Run Configurations to start programs. These are pre-built by ApiLogicServer create.

For VSCode, start the server with F5 and stop with Shift-F5 or the red stop button.

For PyCharm, start the server with CTL-D and stop with the red stop button.

3. Entering a New Order

To enter a new Order:

1.     Click `Customer 1`

2.     Click + ADD NEW ORDER

3.     Set Notes to "hurry", and press SAVE AND SHOW

4.     Click + ADD NEW ITEM

5.     Enter Quantity 1, lookup "Product 1", and click SAVE AND ADD ANOTHER

6.     Enter Quantity 2000, lookup "Product 2", and click SAVE

7.     Observe the constraint error, triggered by rollups from the Item to the Order and Customer

8.     Correct the quantity to 2, and click Save

4. Update the Order

To explore our new logic for green products:

1.     Access the previous order, and ADD NEW ITEM

2.     Enter quantity 11, lookup product Green, and click Save.

Appendix: Add Database Column

The database here is SQLite. You can use the SQLite CLI to add a column using the terminal window of your IDE:

$ sqlite3 database/db.sqlite

>   alter table Products Add CarbonNeutral Boolean;

>   .exit

The SQLite DBMS is installed with API Logic Server, but the CLI is not provided on all systems. If it's not installed, you can install it like this.

We Provide consulting, implementation, and management services on DevOps, DevSecOps, DataOps, Cloud, Automated Ops, Microservices, Infrastructure, and Security

Services offered by us: https://www.zippyops.com/services

Our Products: https://www.zippyops.com/products

Our Solutions: https://www.zippyops.com/solutions

For Demo, videos check out YouTube Playlist: https://www.youtube.com/watch?v=4FYvPooN_Tg&list=PLCJ3JpanNyCfXlHahZhYgJH9-rV6ouPro

 If this seems interesting, please email us at [email protected] for a call.

Relevant Blogs: