GitHubLive AppChat

Building Glue Stack From an Empty Folder

This covers how to create an application that manages users from a blank folder. It won't be kept quite as up to date as the Development Process Document because its a lot longer. It was interesting to go through and rewrite the application I just wrote and clarify some of the stranger parts. I think it would be really cool to show new team members how to build the core framework of the existing application from scratch but the cost might outweigh the benefit.

  1. Create a file to write down any questions you have or anything that's unclear in this document so that we can improve it to make it easier for the next person.

  2. Run the setup.sh as part of the Running Locally process. This will install some of the tools we're going to need.

  3. Make a development folder in your home folder. This is for all your development stuff.

    1. Open terminal. Command + Space (Spotlight) then type terminal.

    2. It will open your home folder by default, indicated by the ~ (tilde).

      cd ~
mkdir development
cd development

  4. Create a folder for our app.

     mkdir glue-stack
 cd glue-stack

  5. Initialise the folder as a git repository. Git is a tool for version control which tracks changes to files.

     git init

Database

Connect to MySQL

We're going to download MySQL and run it using a tool called Docker Compose which uses Docker then connect to it using Sequel Pro.

  1. Create a docker compose file.

    1. Use vim to create a docker-compose.yml file.

    2. Hit i to insert text.

    3. Paste the following:

    4. Finish inserting by hitting Esc.

    5. Save and quit using :wq (colon then w then q, not all at once).

      The standard port for MySQL is 3306 but I've chosen to map it to 3307 so it doesn't clash if you already have a MySQL database running - you can't have two applications listening to the same port.

  2. Start the database.

    The MySQL database will be open after the lineStarting MySQL 5.7.21-1.1.3. To close it hit Control + C.

  3. Commit your changes to git from terminal.

    1. Stage all your files (will just be our docker-compose file).

    2. Commit.

      This will bring up a vim editor just like we did before. To write your commit message you use i to insert, write your commit message "added docker-compose file for database" above the commented (hashed) lines, then esc, then :wq.

  4. Download and install Sequel Pro from https://www.sequelpro.com/.

  5. Open Sequel Pro (possibly using spotlight).

  6. Connect to the MySQL database.

  7. Click test connection.

  8. If it succeeds click "Add to Favourites".

  9. Click "Connect".

  10. Select our "glue" database using the dropdown on the top left.

Create Tables

Organisation Table

  1. Click the plus on the bottom left.

    Name: organisation (its considered good practice to make sure table names aren't plurals; they're singular)

    Table Encoding: UTF-8 Unicode (utf8mb4) (Unicode let's you store characters from all different languages)

    Table Collation: Default (utf8mb4_general_ci)

  2. Click Add.

  3. That should bring up the Structure tab where you can manage the Fields of the table.

  4. Click the plus icon in the middle of the page underneath the lone id Field to add a new field.

  5. Add another Field.

  6. Add another Field.

  7. Add another Field.

Cool you just created the organisation table! If you right click on the organisation table then go "Copy Create Table Syntax" it should look like this:

Alternatively, you can click the console button on the top right to see the SQL statements Sequel Pro has ran then you can filter for "create" and "alter table" statements. This is useful if you've modified an existing table.

User Table

Each user belongs to a single organisation. We call this a Many-To-One relationship from a user to an organisation (or a One-To-Many relationship from an organisation to users). We model this relationship in relational databases using a foreign key. I like to think of relationships as associations as in a user is associated with an organisation and an organisation has users associated with it. The user table also has unique email addresses - no two users can have the same email.

  1. Click the plus on the bottom left again, name the table task with utf8mb4 again.

  2. Add the organisation field.

    The type of this column is going to match the type of the id of the organisation table. To keep things consistent all id columns are the same type.

  3. Add a foreign key.

    A foreign key is a relational database constraint that means a particular field must match another field. In this case we're making sure the organisationId references an existing organisation.

    1. Click the Relations tab (for the user table).

    2. Click the plus icon.

    3. Click Add.

  4. Add another Field.

  5. Add a unique index to the email field.

    This makes sure that two users can't have the same email address.

    1. On the structure tab, click the plus at the very bottom of the indexes panel.

    2. Click Add.

  6. Add another Field.

  7. Add another Field.

  8. Add another Field.

  9. Add another Field.

  10. Let's add the active, createdDate, modifiedDate fields but since we did the same thing for the organisation table we can click the console icon, filter for "alter table" statements and copy the ones for those fields. Open up the query tab and paste them in there and modify them for the user table.

Highlight them all then hit Command+r to run them.

That's it! Have a play with the content tab on the tables to enter data if you're keen.

API

Get the API Running

  1. Install the Spring Boot CLI source.

  2. Init a new spring boot project source.

    • Spring Actuator lets you access metrics about how the application is running

    • Spring Data JPA is a convenient way to access data

    • Spring Boot Development Tools gives us automatic reloads

    • Flyway keeps the database in sync with the application by managing SQL update scripts

    • MySQL is the Java driver for the MySQL database

    • Spring Security is for securing endpoints by requiring authentication

    • Web is for web applications and includes Tomcat and Spring MVC

    You can find a full list of possible dependencies by running

  3. Setup VSCode.

  4. Open the glue-stack folder.

  5. Add some configuration to our project.

    1. Hit Command+p then start typing application.properties then open it.

      This is where we configure our Spring Boot application.

    2. Add the following:

    3. Save using Command + s.

  6. Hit Control + ~ to bring up a terminal.

  7. Run the app for the first time!

    It should finish with Started ApiApplication in x seconds.

    You don't have to remember the command every time because you can use Control+r to search for previous commands. So you can hit Control+r then type spring to search for commands containing spring. You can then use Control+r again to search further back or Control+s search forwards.

  8. Open Chrome and go to http://localhost:8080/.

  9. Notice how everything is secure by default.

  10. Commit your changes using the second menu item on the left. Just like before you can stage all or individual files and write a commit message saying "first running spring boot commit".

  11. Configure Spring Security.

  12. Open ApiApplication.java (use Command+p).

  13. In that folder create a folder called config. This will be a package in Java.

  14. Create a file in config called WebSecurityConfiguration.java.

    The @Configuration annotation means spring will use this to configure the application.

    This configures Spring Security to allow all request to all endpoints.

  15. Save using Command + s.

  16. Go to http://localhost:8080/actuator/health and it should say say "status: up".

    This is an Actuator endpoint.

Add Database Scripts

  1. Create a file called V1__create.sql at api/src/main/resources/db/migration (you'll need to create a db and migration folder).

  2. Go back to Sequel Pro.

  3. Right click the organisation tables, click "Copy Create Table Syntax" then paste it into the file.

  4. Do the same for the user table then the task table.

    Its important to do it in this order because the task table depends on the user table which depends on the organisation table.

  5. Save using Command + s.

  6. Delete the task table then the user table the organisation table.

    Again, the order is important because of the dependencies.

  7. Hit Control+c in the terminal where the API is running to stop it.

  8. Open up the application.properties file.

  9. Remove the comment and replace the last line with.

  10. Save using Command + s.

  11. Start the API again by hitting the arrow keys up and down to find the command you previously used to start it.

  12. Go back to Sequel Pro and hit the refresh button on the bottom of the left pain to refresh the tables.

    You should see all your tables re-appaear plus a new table called flyway_schema_history. That table is what Flyway uses internally to check where your database is at so it can run scripts on startup so that everyone's databases is kept up to date.

  13. Commit your work saying "added database setup scripts".

Create JPA (Hibernate) Entities

So we've now defined our database tables and their relationships to one another but we still need to create Java classes to represent that data so that we can use them effectively in our Java application. These Java classes are called Entities.

JPA (Java Persistence API) is a specification for Java ORM (Object Relational Mapping). A specification is like a Java interface where you define how you interact with the API without implementing it. ORM is all about mapping relational data in our database to Object Orientated data in our application so the data is more convenient to navigate. Some people really don't like ORM and I can definitely relate but I believe it's a useful concept to know. Hibernate is our JPA implementation that makes the JPA specification work and was around before JPA so that's why some sources will talk about hibernate without mentioning JPA.

Earlier, we identified some columns were the same in all the tables; id, active, createdDate, modifiedDate. We can also see that both the user and task table have an organisationId. Let me introduce you to the first programming concept in this tutorial; DRY - don't repeat yourself. This means try not to duplicate (copy and paste) code. Some people are pretty good at identifying common code in their head but i've always found it easiest to write duplicated code first so I can easily identify the common parts then abstract them. Abstraction means to reduce duplication.

I'm going to walk you through making the abstractions first becuase it makes this tutorial easier but I encourage you to write each of the classes in full then have a go abstracting second because I that's how I would develop outside this tutorial.

  1. Create a file called BaseEntity.java at api/src/main/java/org/gluestack/api/domain/entity (create the domain and entity folders (packages)).

    Cool, what we've done is declare the id, active, createdDate and modifiedDate as fields and annotated them with some extra information for Hibernate. In Java the convention is to use getters and setters for each field instead of making them public so underneath the fields start writing set and get for each field and then you can use the autocomplete result. This is easier in other editors and I won't repeat this process for the rest of the tutorial.

  2. Create a BaseOrganisedEntity.java in the same folder and generate the getter and setter for the organisation field. Don't generate the getter and setter for the superclass (BaseEntity) fields. Don't worry about compilation errors until after we've created all our entities because they all refer to each other.

  3. Create Organisation.java. This is largely what you would call a POJO in Java - Plain Old Java Object. A POJO is Java class that just has fields and getters and setters. We've added some annotations but I still largely consider this a POJO.

  4. Create User.java. This one is a little bit more involved because it's the implementation for our user for Spring Security.

  5. Start the application. Hibernate will validate that your data classes match your database and will fail to start if they don't.

  6. Commit your work; "created entities".

Create the Repository Layer

The repository layer in Spring is a layer dedicated for communicating with data sources.

To do this, we'll need to add some dependencies to our project. Dependencies are dependencies on libraries. A Library is a separate, reusable piece of code that does something and makes your life easier and associated with the concept of abstraction which means to reduce complexity. A lot of people write Java Web Applications so its only natural that we deal with the same problems so libraries are an attempt to solve some of those common problems. In Java you used to have to download a compiled library but this is hard to maintain and leads to large project sizes so we use what's called dependency management where you simply specify the library and the version that you want and that will be used to download that library for you. In Java you can use Maven to manage dependencies and it stores the version information in a pom.xml file.

  1. Add Querydsl JPA as a dependency. Querydsl JPA is a Java library for typesafe JPA queries that also works really well with another one of our cool libraries, Spring-Data-JPA, which is powering our repository layer.

    1. Open the pom.xml file (using command + p) and add the following line to the properties element.

    2. Add the following dependency below Flyway.

    3. Add the following plugin to the plugins element below the spring-boot-maven-plugin.

  2. Add this plugin so VSCode finds the classes that Querydsl generates.

  3. Create the BaseRepository.java file at api/src/main/java/org/gluestack/api/repository.

  4. Create the OrganisationRepository.java.

  5. Create the UserRepository.java. The findOneByEmail method will allow us to find a User in the database by their email and the EntityGraph annotation will load their Organisation in the same query. This will be useful for authentication. This is where Spring Data JPA is pretty cool, its an interface that automagically implements itself!

  6. Restart the application to make sure its all correct.

  7. Commit your work; "created repository layer".

Create the Service Layer

The service layer is where most of your API logic lives. Some people choose to split up the service layer further where they see fit but I prefer Services call other Services as required instead of creating mandatory layers.

  1. Create the BaseService.java file at api/src/main/java/org/gluestack/api/service. Its a lot of custom code that I've developed for this project so just copy and paste it from this repo; BaseService.java. You may have to update the package and some of the imports so it compiles.

  2. Create the OrganisationService.java file in the same folder.

    This demonstrates the general structure of the services:

    There's a method that returns a Querydsl Predicate which is a typesafe database criteria that has defines what entities can be returned from the database. You can see that this will only match the organisation of the current user and this is repeated for the other services as well.

    The create method for an organisation also needs to create a user which this service is not responsible for so it uses a method on the UserService.

  3. Create the UserService.java.

    In this Service it overrides one of the save lifecycle methods to implement some extra functionality; hashing a user's plaintext password. The hashing functionality is provided by Spring Security and is important for keeping passwords safe in the database because it is a one way hash, it can't be decrypted. The API validates user's passwords by applying the hash to the password supplied during authentication and is checked against the hash in the database. The only way to 'decrypt' the hash is to generate hashes which should take a long time. That's why new hashing algorithms are created to stay ahead of the speed improvements of computers.

    Java has a convention of Mapping which might help here but I don't like the pattern much so i'm trying to avoid it and I think this code is okay.

  4. As always, start the appliation to make sure it works then commit your changes.

Create the Controller layer

The controller layer is what connects our application to the internet. It defines what URLs the application listens to and what Java objects we're expecting to receive or send. Spring MVC will convert the Java objects to and from JSON which is convenient for the frontend.

  1. Create the BaseController.java at api/src/main/java/org/gluestack/api/controller. Again, like the the BaseService its custom code so copy and paste it; BaseController.java.

  2. Create the OrganisationController.java. This controller is a bit unique because it needs to handle new signups which means it works without a logged in user.

  3. Create the UserController.java. This is what our typical controller method looks like, I would have liked to move the findAll method to the BaseController but currently its not possible.

  4. Create the AuthenticationController.java. This is what the frontend will call to login and if authentication is successful it will return the details of the current user.

  5. Run the application and commit your work.

Final Configuration

  1. Add the dependency for Jackson Hibernate Module inside the dependencies element in your pom.xml file.

    In Java web applications there is the concept of mapping where you convert your Entities into other Java objects which the API would return. Instead of doing that I prefer to use my Entities for the output but Jackson, the library that spring uses to convert Java objects to and from JSON (serialisation and deserialisation), would try and serialise every relation of each entitity (and their relations and so on and so on), causing an infinite loop. This library prevents this by making Jackson only serialise the entities that have been loaded from the database. That means if you want to return more data all you have to do is load more data from the database, which you would have to do anyway if you mapped entities.

  2. Create WebMvcConfig.java inside the config folder.

    This will enable the Jackson Hibernate Module and the Spring Data Web Support which converts URL parameters into a Querydsl predicate to support filtering.

  3. Create the UserDetailsService.java. This will be used to configure Spring Security and will be used as part of authentication.

  4. Open up the WebSecurityConfiguration.java file. Make it look like this:

  5. Start the application and commit your work.

Tests

I'm not the best at tests but I think I like this approach.

  1. Add the dependency on Testcontainers in the pom.xml.

    Testcontainers allow us use a real MySQL database for our tests and its loaded using docker so you have to have that installed but luckily that's part of our setup scripts.

  2. Create an application-test.properties at api/src/main/resources/.

    This will specify to connect to the testcontainers database during testing. You don't need to replace anything in this code snippet.

  3. Delete the ApiApplication.java file from api/src/test/java/org/gluestack/api.

  4. In that same directory, create the BaseTest.java.

    My current testing strategy is to have a decent amount of test data available so that you don't have to setup a lot of data for new tests. This might not scale in the long run so I may have to break it up a bit but I think the concept of making it really easy to setup data or not having to do it at all is a good.

    The other important thing here is the Transactional annotation. That each test will run in a separate database transaction and will be rolled-back after the test completes. This means the database will be in the same state before each tests regardless of which tests ran before it, making our tests independent.

  5. In that same folder create TestOrganisation.java. This is the object to pass around test data.

  6. Create TestOrganisationService.java. This creates the test data.

  7. Open BaseTest.java and add the following to the bottom of the class. This will setup two organisations and their data so that we can reference it for our tests.

  8. Create a POJO for the PageResponse the list views return. I'm not 100% sure why we have to do this but its not terribly hard. Copy it from https://github.com/cadbox1/glue-stack/blob/master/api/src/test/java/com/api/PageResponse.java

  9. Create CreateTest.java at api/src/test/java/org/gluestack/api/organisation

  10. Create FindAllTest.java at api/src/test/java/org/gluestack/api/user

  11. At the command line, inside the api folder, run this command to run all the tests.

  12. Commit your work.

UI

Create React App

  1. Open a new terminal in VSCode and run this command.

    This will install create-react-app globally. create-react-app is by far the most popular way to get a React project started.

  2. Create a new project by running the command then the name of the project.

  3. Follow the commands it displays in the terminal.

    That should open a working react app in your browser, how easy was that!

  4. Add this line before the dependencies.

    This will route any unkown urls to our API.

  5. Stop the front end development server using Control + c.

  6. Run yarn. This will produce a new lockfile.

    Yarn is an alternative package manager to npm that uses a lockfile that records the exact version of each dependency that is installed. The lockfile differs from that package.json which stores the ranges of versions that are to be installed or updated to. e.g do you want to jump major versions or just minor versions. The lockfile means more consistent results across different environments.

  7. Commit your work.

  8. create a .env file at ui/.

    This will allow us to import the folders inside the src folder absolutely instead of relatively which is more robust when moving stuff around. Its always a good idea to link any relative issues in code that help you come to a solution.

  9. Create a .prettierrc.yaml file.

    This will specify how we want the prettier to format our code, including the prettier VSCode plugin we installed.

  10. Create a tsconfig.json file.

    This will let VSCode understand our project. Tt will make the autocompletes, called Intellisense in VSCode, a lot more helpful.

  11. Commit your changes.

Add Libraries

We're going to add some Libraries to our project mainly in the form of dependencies. Just like in Java we're going to have a file that holds our library names and versions so that we can download them all instead of putting them in our project. When we run yarn add it will download the library from NPM and add it to our package.json.

  1. Open a terminal window and cd into the ui directory.

  2. React Router (and friends). This is for Routing which is about mapping the url in the address bar to different pages in our app.

  3. Axios is a small library that makes netwrok requests just that much easier.

  4. Material UI Beta. Material UI is the most popular material design library for React. This library is going to heavily influence the design of our app using the popular Material Design specifications. Their previous version had some pretty big limitations once you got into it but they've learned a lot and delivered a fantastic beta version. Being a Beta they tend to Move Fast and Break Things (and that's a good thing) so we're going to use the exact version I know works.

  5. Bootstrap. Okay, we definitely shouldn't need this but I know the utilities fairly well so it helped me get the ui working quickly but I will definitely be removing this eventually.

  6. React Component Queries allows us to detect the sizes of components so that we can keep our app responsive and working nicely on mobile devices.

  7. Okay this next step is a bit of a cop out. Its the re-usable code I've developed while creating the frontend. I really should be explaining each peice but I think i'll come back and do that later. Sorry!

  8. Copy the common folder to ui/src/.

  9. Start the frontend server. This will automatically open the app in a browser window.

  10. Setup React Router and Material UI at the root component. Open App.js.

  11. The browser window should automatically refresh and display the word "App".

  12. Delete the logo.svg.

  13. Open App.css.

  14. Commit your work.

API Requests

  1. Create a folder called api at ui/src/.

  2. Create the organisation.js file.

  3. Create the user.js file.

  4. Create the task.js file.

  5. Create the authentication.js file.

  6. Commit your work.

Authentication Component

  1. Create a folder called main at ui/src/.

  2. Create the index.js file at ui/src/main.

  3. Open App.js.

  4. Delete the <p>App</p> line and start typing the word <Main. Then hit enter on the first autocomplete result that says "import main". This will automatically add the import for the Main component.

  5. Close the Main component tag with /> so it says <Main />.

  6. The browser window should now show "Main".

  7. Commit your work.

Login Component

  1. Create the index.js file at ui/src/main/unauthenticated/login.

    class Login extends Component { constructor(props) { super(props); this.state = { email: "", password: "" }; }

    }

    Login = withRouter(Login);

    export { Login };

    ```

  2. Create the index.js at ui/src/main/unauthenticated.

  3. Open the main/index.js by hitting Command + p then start typing main then type /.

  4. Add this between the start of the render function and the return.

    So it looks like this.

  5. Put yor cursor at the end of Unauthenticated word then hit Control + space to bring up the autocomplete. Select the option to import the Unauthenticated component, it should be the second option.

  6. The browser should now be showing a login form.

Signup Component

Notice how our Signup Here link doesn't work. Let's fix that.

  1. Create the index.js file at ui/src/main/unauthenticated/signup.

  2. Open unauthenticated/index.js using Command + p.

  3. Before the <Route>, create a route for the signup component.

  4. Again, use autocomplete to import the signup component. I'm not going to say this anymore you can just do it for new stuff.

  5. The signup link should now work as well as the login here link on the signup page.

First Signup

  1. Start the API and Database if they are not already running.

    1. In separate tabs in VSCode.

    2. Database.

  2. API.

  3. Fill in the signup form and click signup. You should now see the word "Main" which means you are logged in.

Sidebar

  1. Create index.js at ui/src/main/authenticated/sidebar.

  2. Create a drawer.js file in the same folder.

  3. Create index.js at ui/src/main/authenticated.

  4. Open main/index.js and replace return "Main" in the render function with:

  5. You should now be able to logout, toggle the sidebar and see how it behaves differently on smaller screens. None of the links change anything but we'll fix that soon.

  6. Commit your changes.

Users

  1. Create the list.js file at ui/src/main/authenticated/user/.

  2. Create the form.js file in the same folder.

  3. Create the index.js file.

  4. Replace the button in the authenticated/index.js file with these routes.

  5. Add the react-router imports and use the auto import for the User component

  6. Commit your work.

PreviousSetup Database Connection In Sequel ProNextDeveloping Process Documentation

Last updated