Sarapis

This notebook will have Sarapis related resources

Sarapis Architecture Design

Overview

Sarapis has an open-source backend data administrator interface tailored for managing Human Services Data Standard (HSDS) datasets, which provide standardized information about health, human, and social services. The project's primary objective will be to create a modular, service-based backend that simplifies the management, validation, and interaction of HSDS datasets. 

Project Goals

Key Features

High-Level Architecture Overview

The backend service will be hosted and deployed to AWS. Therefore, we can leverage existing AWS services to achieve some of the key features mentioned in the project specification. The service does not require scalable services for each feature, so the Micro-service architecture is discarded. Instead, the backend service will follow a monolithic architecture where all the endpoints will be published in a single service. 

AWS Lambda

AWS is a serverless computing service that lets you run code without provisioning or managing servers. Creating an HTTP server using Lambda involves integrating Lambda with Amazon API Gateway, a managed service that allows you to create, publish, maintain, and secure RESTful APIs. This setup enables Lambda to handle incoming HTTP requests, effectively simulating an HTTP server while leveraging the benefits of a serverless architecture.

image.png

Core components 

Lambdas can also be integrated with AWS Cognito for token-based authentication, or you can also establish custom Lambda authorizers for more specific access control logic. 

Since Lambda is inherently stateless:

Limitations and Challenges

  1. Cold Starts: If not used recently (i.e., a "cold start"), lambda functions may incur latency on the first invocation, impacting applications with strict performance requirements.

Elasticsearch

Elasticsearch serves as a search engine. It is integrated primarily for its ability to index and query, large volumes of data quickly. In this architecture, Elasticsearch complements DynamoDB by providing efficient querying capabilities, particularly for complex searches that are not natively supported in DynamoDB.

Data Ingestion and Synchronization:

Query Operations:

Challenges with Elasticsearch Integration

Appendix

What is a taxonomy?

You can think of a taxonomy as a structured system used to classify and organize information into categories, making it easier to understand, manage, and retrieve. It involves some sort of hierarchical arrangement, where items are grouped into broader categories and further divided into subcategories.

Health Services
- Medical Care
- - Primary Care
- - Specialty Care
- Mental Health
Housing Services
- Emergency Shelters
- Affordable Housing
- Housing Assistance Programs

Spring Boot on EC2

Spring Boot with EC2 is a traditional setup for deploying a self-contained backend application on virtual machines. The backend application can be packaged as a standalone executable using a Java build tool such as Maven or Gradle. Spring Boot offers dependency management, pre-configured settings, and integrated database support, making it ideal for creating microservices and RESTful APIs.

The advantage of choosing Sprint Boot is its more flexible runtime. There are no cold starts. The application can be stateful by attaching persistent storage to an EC2 instance. 

Limitations and Challenges

Microservice Architecture

Due to the scalability and flexibility requirements of the project following a micro-services architecture, it will align well with the goals of modularity, scalability, and integration within the broader open-source ecosystem. The microservice architecture for the proposed HSDS backend administration interface can be structured as follows:

You can view the HSDS schema here.

image.png

Managed Pipeline

With large projects such as Sarapis with multiple components, it is important to define a process for testing and releasing new versions. As such, we are proposing a managed pipeline that automates testing, health checks, and deployments to different evironments. These environments would include:

The pipeline should also allow Blueprint to easily rollback versions on the Software. Due to limited resources, we are looking to build this rapidly with minimum Infrastructure cost. A viable tool that we can use to fit this objective is GitHub actions, that way are infrastructure remains close to the code. We may have to use additional tools like Jenkins to handle builds, but that is what this document will define/explore. 

A Minimum Viable Product Might Include:

10/9 Meeting Minutes

Multiple Repos:

image.png

Manage integrations through:

Non-Profit Meeting

Ownership:

Jonathan: User Management Service

Terrence: Data Management Service

 

Devin Meeting:

Sarapis Development Resources

HSDS Reference

https://docs.openreferral.org/en/v2.0.1/hsds/reference/http://docs.openreferral.org/en/latest/hsds/schema_reference.html

Tutorial: Create a CRUD HTTP API with Lambda and DynamoDB

https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-dynamo-db.html

Realize A Simple CRUD API on Amazon DynamoDB

https://medium.com/@guraycintir/realize-a-simple-crud-api-on-amazon-dynamodb-1457a5e124b7

API Gateway Lambda DynamoDB CRUD Flow Repo

https://github.com/gcintir/api-gateway-lambda-dynamodb-crud-flow/tree/main

AWS CDK

https://docs.aws.amazon.com/cdk/v2/guide/hello_world.html

Testing with SAM

https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-cdk-testing.html

Locally invoke Lambda functions with AWS SAM

https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-using-invoke.html

SAM invoke documentation

https://awscli.amazonaws.com/v2/documentation/api/latest/reference/lambda/invoke.html

Key condition expressions for the Query operation in DynamoDB

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Query.KeyConditionExpressions.html

Create an Amazon Cognito user pool for a REST API

https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-create-cognito-user-pool.html

Add more features and security options to your user pool

https://docs.aws.amazon.com/cognito/latest/developerguide/user-pool-next-steps.html

AWS Cognito with ReactJS for authentication

https://medium.com/@adi2308/aws-cognito-with-reactjs-for-authentication-c8916b873ccb

Use API Gateway Lambda authorizers

https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-use-lambda-authorizer.html

DynamoDB Local Sample Java Project

https://github.com/awslabs/amazon-dynamodb-local-samples

Configure the URLConnection-based HTTP client

https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/http-configuration-url.html

DTOs

https://medium.com/linkit-intecs/getting-started-with-crud-operations-in-spring-boot-and-dynamodb-a-beginners-guide-75ecad3b0452

Amazon Cognito

Integrating Amazon Cognito for Authentication and Authorization in a Spring Boot Application

https://medium.com/@abhishekranjandev/integrating-amazon-cognito-for-authentication-and-authorization-in-a-spring-boot-application-fe5fe7d78db

The Easiest Way to Deploy Containers on AWS

https://www.fernandomc.com/posts/easiest-way-to-deploy-aws-containers/

Spring Boot Role-Based Authentication with AWS Cognito

https://howtodoinjava.com/spring-security/spring-boot-role-based-authentication-with-aws-cognito/