Executive Summary
In this project you will create a number of multithreaded TCP services as building blocks of a micro-service architecture. Using a custom (non-HTTP) protocol. Tthese services will expose a number of foundational concepts, such as basic networking, json/xml marshaling, database access, inter-service calls, stateful vs restful, and cloud computing. You will later use these services to build a shopping-cart web application. In addition, this project will address scalability issues such as multithreading, connection pooling, and throttling, as well as certain security concerns such as vulnerabilities and authentication.The Environment
All development work in this course must be done within your home directory on EECS systems; i.e. not on your own computer. You can access your home directory by visiting the remoteLab URL and then selecting an EECS system to work on:- During the scheduled lab hours, select any available workstation under Linux Remote Desktop (EDU) (ea).
- Outside the the scheduled lab hours, select red1 or red2 under Linux Remote Desktop (EDU) (red).
Note that it is also possible to access your home directory at any time by connecting to red.eecs.yorku.ca via ssh but this gives you CLI access only.
To set up your environment for this course, issue the command:
4413GoIt will create a folder named
4413
under your home directory
and it contains a number of libraries, drivers, softwares, and tools. For
development, you can use any editor (nano, vim, ...) or Eclipse to build
your classes, but whatever you use, do not store your own files
(e.g. your workspace) in the 4413 folder. Store them anywhere else under
your home directory and leave 4413 alone.
The Services
This Project asks that you develop and test five micro services with the following functional specifications:- Geo—A Computational Service
Given two points on Earth, this service returns the geodesic distance between them. This service will be used later by a shopping cart application to estimate the drone delivery time. Each point is specified using its latitude (t) and longitude (n) expressed as signed decimal degrees with East of Greenwich being a negative longitude and south of the Equator being a negative latitude. The distance (in km) between two such points is given by:12742 * atan2[sqrt(X), sqrt(1-X)] where X = sin2[(t2-t1)/2] + Y * sin2[(n2-n1)/2] and Y = cos(t1) * cos(t2)
Note that the four coordinates must be in radians so start by multiplying each byπ/180
. Test your code by computing a known distance (you can for example use a map app and pick two points connected by a straight road). Note that we are estimating the drone delivery time here, but in a later project, we will estimate the driving time by taking the road network and the current traffic volume into account. - Quote—A Database Service
Given the ID of a product and a return format (json or xml), this service looks up the ID in the Product table in thehr
schema of the Derby database and returns the product's ID, name, and price in the desired format. If the ID is not found, the return should have "<ID> not found" as ID, an empty name, and 0.0 as price. Use GSON and JAXB to serialize and marshal, and use "id", "name", and "price" as identifiers for the json elements and XML nodes. Create a "Product" bean to facilitate the transformation (more on this in lecture). - Auth—An API & Database Service
Given a username and password, this service authenticates these credentials and returns "OK" or "FAILURE" accordingly. This service will be used later by the shopping cart application to authenticate users. Authentication is done by adding a long salt to the password; using a cryptographic function to hash the result; and then repeating the process count times. More on this in lecture. The CLIENT table in the Sqlite3 database stores the salt, count, and hash of each user. The table adopts PBKDF2 (Password-Based Key Derivation Function 2) to perform the hash, which is the current best practice. An API for computing PBKDF2 is provided in the hr4413 library (in 4413/lib) through the following method in theg.Util
class:public static String hash(String password, String salt, int count) throws Exception
If the username is not found in the table, or if found but the computed hash differs from the stored one, return "FAILURE"; otherwise, return "OK". - Geo2—A Stateful, Delegating Service
This "pedagogical" service delegates to the restful Geo service in a stateful manner to shed light on session management. It receives the coordinates of the first point in one request and the coordinates of the second point in another. It needs to somehow "link" the two request (despite the multithreading nature of the service) and then supply all four real numbers to Geo and return its response (i.e. no computation is involved).
Hint: if only two parameters are received then treat this as a fresh request; store the parameters in some data structure and return a pointer to them as a cookie. If three parameters are received then the third as a cookie that allows you to link this request with a prior one and thus obtain all four coordinates. - Gateway—An HTTP API Gateway
This "pedagogical" service sheds light on the challenges involved in building an API Gateway, such as service registration, discovery, and orchestration. It is also our first foray into HTTP services. Its client is a browser and its URL is:http://host:port/SRV?p1=v1&p2=v2...
where SRV is either Geo or Auth or Quote and where v1, v2, ... are the parameters of the SRV service. The job of this service is to discover the needed service (extract its name from the request and find its IP and port if alive); perform inter-protocol transformation; invoke the service; and then return its response.
All five services use TCP for transport; they are all restful except for Geo2, which
is stateful; and they all use a custom protocol except for Gateway, which uses HTTP.
Service Implementation
Use the design patterns, methodologies, and hints demonstrated in lecture in order to speed up development and learn best-practice approaches.- Develop all services in Java and include 4413/lib in your classpath. If you use Eclipse, configure the project's build path so that its libraries include the needed files in 4413/lib as external jars.
- Each of the 5 micro services must have its own class and its own port. This way, they can be deployed in one container, in different containers, in different VMs, or even on different physical hosts.
- Name the service classes
Geo
,Quote
,Auth
,Stateful
, andGateway
. - The service class must extend
Thread
and must have a main method. - The service class must act as a server with a TCP transport and must bind to localhost.
- The server's port (chosen through a command-line argument) is either zero, to indicatethe any available port, or the (punched) port assigned to you (more on this below).
- The server must be multithreaded with a new thread per connection.
- The custom protocol for the first 4 services is text-based, case-sensitive, and line oriented. The request must be a single line (EOL terminated) with its tokens whitespace delimited. The response is also text-based and EOL terminated.
- For the Gateway service, the protocol is http. Hence, the request
is multi-line and begins with the line:
GET /SRV?p1=v1&p2=v2 HTTP/1.x
followed by HTTP headers that end with an empty line. The response begins with the two lines:HTTP/1.1 200 OK Content-Type: text/plain
followed by an empty line and then the return of the service. - All services return must be "Do not understand <request>" if the incoming request is malformed.
- Use standard out for logging.
- Log "Connection from <ip:port>" upon receiving a request.
- Log "Closing <ip:port>" upon closing a connection.
- Close the connection with the client after issuing the response.
- Handle exceptions per client by logging the exception message and closing the connection with that client.
Testing & Deployment
Do all your tests on the same host; i.e. launch the service in the background (using&
) and test it in the foreground. This way, you
will not be blocked by the York firewall which plugs all non-standard ports.
To test, use
"telnet <ip> <port>
" or one of the tools discussed
in lecture.
Once everything is working, test from outside York (e.g. from your phone or
home machine) by launching the service on red (rather than a workstation)
by simply ssh'ing to it before launching. For the port, use the punched port
assigned to you in this course (to find it, visit ePost from the GRADE page).
Clean-Up
Do not leaves services running unnecessarily in the background on time-shared servers--they will consume system resources. Once you are done with a service, terminate it. Use thefg/bg
commands or ps -u
and
kill
to list and terminate all you services on red, red1, and red2.
Persist Your Work
Locate all your files (i.e. java, class, shell, and other files that you created to build and test this project. Add also aREADME.txt
that briefly describes
what the files are and how to launch them. Upload all these files to this
project's section in the
course cloud. This ensures the files are available to you when you
need them. You can of course also save your work to your favorite git, Google
Drive, DropBox, S3, or some other storage.