Category: Java

Java (JVM) Memory Model

 

What is Java Memory Model and Why it is Important?

The Java memory model specifies how the Java virtual machine works with the computer’s memory (RAM).

It is very important to understand the Java memory model if you want to design correctly behaving concurrent programs. The Java memory model specifies how and when different threads can see values written to shared variables by other threads, and how to synchronize access to shared variables when necessary.

 

Java Memory Model : Heap and Stack

Internal memory model in java, divides the JVM memory into two areas.

  • Heap
  • Stack (known as Thread Stack)

The heap is where all the objects live and the stacks are where the threads do their work. Please refer the below diagram.

java-memory-model-1.png

There is a single heap per JVM and conceptually, there can be multiple thread stacks depending on the number of threads, but it can be limited by the OS based on the available resources (such as memory). Thread stack is local for each thread and heap is shared among all threads.

 

Heap

Heap is the area in the memory where the objects related to (created in) the program resides and lives. all the objects lives in the heap regardless of its nature and creator (creating and owning thread). It is no matter who(which thread) created the object and whether it is static, non static or method local (any scope). All the objects are stored in the heap until they are collected/claimed by the GC (Garbage Collector).

If you new to java, you might be questioning yourself about what will be treated as object?  Every variable reference and assignment that does not belong to any primitive type available (There are eight primitive types)  in java can be considered as objects. Therefore any object instantiation including Wrapper classes (Object version of primitive types e.g:- Byte, Short, LongFloatCharacter, Integer, Double, Long) will also fall into this category and lives in the heap.Therefore the heap may contain any java object that is created because of the execution of the java application.

In addition the heap stores the static variables (static primitive variables) with the class definition.

 

Stack

Each thread running in the JVM (Java virtual machine) has its own thread stack. This stack is private/local to each thread. The thread stack contains information about which method is currently executed and up-to which line it has been executed to reach the current point of execution. That means that each thread has a pointer into the code which points to the current position of code they’re currently running. This is known as the “call stack“. As the thread executes the code, the call stack may change.

The thread stack also contains all local variables for each method being executed (all methods on the call stack). A thread can only access it’s own thread stack. Local variables created by a thread are only visible to that thread and  No other threads can access them.

local variables :- primitive and object reference (not actual objects) variables that are local for the thread including method local variable, instance variable of the class and object variable references.

Even if two threads are executing the exact same code, the two threads will still create the local variables of that code in each their own thread stack. Thus, each thread has its own version of each local variable. . When a thread starts running a new method it saves the arguments and local variables in that method on its own stack. Some of these values might be pointers to objects on the heap. If two threads are running the same method at the same time they will both have their code pointers pointing at that method and have their own copies of arguments and local variables on their stacks. They will only interfere with each other if the things on their stacks point to the same objects on the heap.

All local variables of primitive types (booleanbyteshortcharintlongfloatdouble) are fully stored on the thread stack and therefore they are not visible to other threads. One thread may pass a copy of a primitive variable to another thread, but it cannot share the primitive local variable itself.

When the method execution is completed, all the local variables and call stacks will be removed from the memory (stack).

 

 

Where the “static” variables are stored ?

Prior to Java 8 (Java 7 and Older versions) static variables are stored in the PermGen (Permanent Generation) space. PermGem space is a special memory area available in the java heap. PermGen space was removed in Java 8 and introduced Metaspace to overcome some limitation of the PermGen space. In Java 8, static variables are stored in the Metaspace.

 

 

What is PermGen (Permanent Generation) and Metaspace?

PermGen space is a special memory area available in the JVM heap with Java 7 and older versions. PermGem was used to store meta information about the classes that has been loaded into JVM.  In addition, PermGem space is used for storing the static variables of the application. The problem with PermGem space is that it has a fixed max size. Since the PermGen space is a part of Heap, it cannot exceed the allocated memory.  Therefore it throws the following error when the it reaches the max PermGem space.

java.lang.OutOfMemoryError: PermGen space

PermGen space was completely removed in Java 8 and Metaspace was introduced as a replacement for the PermGen. Metaspace stores the Class definitions of your Objects, and some other metadata (like static variables).

The main advantage of Metaspace is that it resides in the main memory area of the system (not a part of the Heap) and it does not have a fixed max size. It can increases its size on runtime depending on the available memory of the system. Therefore OutOfMemoryError will not be occurred with Metaspace.

 

 

Consider the following code segment. 

Student std = new Student();

after executing the above line, JVM memory will be like this.

  • Heap:  stores “new Student()” (object that has been created)
  • Stack: stores information about “std” (reference variable)
  • PermGen Space / Metaspace: stores the meta information about Student class

 

 

Here is the diagram that illustrates how the objects are stored in the heap, local variables are stored in the stack and method invocations are stored in the call stack.

java-memory-model-2.png

If the local variable is a primitive type, then it is stored in the Thread Stack,

If the local variable is a reference variable to an object, then the reference variable is stored in the Thread Stack and Object is stored in the Heap.

An object may contain methods and these methods may contain local variables. These local variables are also stored on the thread stack, even if the object belongs to the method, it is stored on the heap. (any object is stored in the heap regardless of its scope)

An object’s member variables (member variables of  instance) are stored on the heap along with the object itself. That is true in cases when the member variable is  a primitive type or  if it is a reference to an object.

static variables (class members) are also stored on the heap (PermGen space or Metaspace) along with the class definition.

Objects on the heap can be accessed by all threads that have a reference to the object. When a thread has access to an object, it can also get access to that object’s member variables. If two threads call a method on the same object at the same time, they will both have access to the object’s member variables, but each thread will have its own copy of the local variables.

 

 

Consider the below code examples.

After executing the above example, the heap and stack of the JVM will looks like below (Before running the Garbage Collector) .

java-memory-model-3.png

There are two threads that execute the MyRunnable instances.  run() method will invoke the methodOne().  Then the methodOne() will invoke the methodTwo(). Therefore the call stack for both method should reside in the each Thread stack.

methodOne() declares a primitive local variable (localVariable1) that store a primitive value and object reference variable (localVariable2) that refers the static object (object3) available in the heap.  The local primitive variable and object reference will be stored in the local thread stack.

If we look at the MySharedObject class, you can notice that it creates a static object (object3) and  two Integer wrapper class objects ( known as object2 and object4). Since all the objects live in the heap, the object3 , object2 and object4 should reside in the heap. In addition, the object3 will have the references for the object2 and object4.

Since the object3 is static, both threads will have the reference for the same object. In other words, both threads will share the object3.

methodOne() will invoke the methodTwo(). if you look at the methodTwo() implementation, you may notice that it creates an Integer wrapper class object which is local to the thread and method. Even if it is local to the thread itself, it is a object creation. Therefore the object should resides in the heap. Since there are two threads running, there should be two another objects available in the heap and each thread stack should have the reference for related object (please see object1 and object5).

References :-

http://tutorials.jenkov.com/java-concurrency/java-memory-model.html

CountDownLatch in Java Concurrency

What is CountDownLatch?

In CountDownLatch, it enables one thread (calling thread) to wait until other threads completes their tasks.

e.g. Application’s main thread want to wait, till other service threads which are responsible for starting framework services have completed started all services.

CountDownLatch works by having a counter initialized with number of threads, which is decremented each time a thread complete its execution (by calling countDown() method available in the CountDownLatch class). When count reaches to zero, it means all threads have completed their execution, and the thread waiting on latch (caller thread)  resume its execution.

 

How CountDownLatch works?

CountDownLatch.java class defines one constructor inside:

public CountDownLatch(int count) {...}

This will create a CountDownLatch that will wait until the given number of thread completes their executions.

e.g:-

CountDownLatch latch1 = new CountDownLatch(4);
latch1.await();

The latch1.await()  hold the currently running thread until 4 threads have completed their execution with latch1. each completion is acknowledged to latch1 with countDown() method.

 This count is essentially the number of threads, for which latch should wait. This value can be set only once, and CountDownLatch provides no other mechanism to reset this count. (You cannot reset or change this value later)

The first interaction with CountDownLatch is made by the main thread (caller thread) which is going to wait for other threads.

This main thread (caller thread) must call, CountDownLatch.await() method immediately after starting other threads. Then await() method will hold the execution of the caller thread until all the given threads are executed. The execution of the await() method will be expired once the count of the CountDownLatch is zero and the caller method will be resumed.

(as i have mentioned earlier, each thread should call the countDown() method on the latch upon completion of the thread. each countDown() call will decrement the count value by one)

 

CountdownLatch_example.png

Lets look at the diagram in detailed.

ThreadA (the main thread) has created a CountDownLatch that should wait for  3 number of threads.

ThreadA has created another three threads and started them. As soon as they are started,  ThreadA call the await() method of the CountDownLatch.

The await() method holds the execution of the caller method (that is ThreadA) until the specified number of threads (In this case three threads) are completed (in the CountDownLatch).

Each thread should call the countDown() method of the CountDownLatch upon completion of its execution.  countDown() method will decrement the count value of the CountDownLatch by one.

As soon as the count is zero, CountdownLatch get to know that all the threads has completed their tasks. Therefore the caller thread (ThreadA) that is waiting on the CountDownLatch will be resumed.

 

Now lets look at the code implementation for the above scenario.

 

WorkerThread has been implemented in a way to sleep for 5 seconds. After that it will print the name property and call the countDown() method of the CountDownLatch.

 

 

After executing the above Application, you will get the following output.

 main thread has started 
 all the worker threads have been started. next calling await() method 
 thread [thread2] completed 
 thread [thread4] completed 
 thread [thread3] completed 
 thread [thread1] completed 
 Resume the main thread after executing of all worker threads

 

 

 

 

Java – Thread : run() or start() method?

If you just invoke run() directly, it’s executed on the calling thread, just like any other method call.

Thread.start() will create a new thread so that the runnable’s run method is executed in parallel.

when program calls start() method a new Thread is created and code inside run() method is executed in new Thread while if you call run() method directly no new Thread is created and code inside run() will execute on currently running (existing) Thread.

Another difference between start() vs run()  in Java thread is that you cannot call start() method more than one time on thread object. once started, second call of start() will throw IllegalStateException in Java while you can call run() method any number of times.

Starting and Running Thread in java

How to create Thread in Java?

There are two ways to create a Thread in java.

  • Create a subclass of the Thread class and override the run() method.
  • Create a class that implements the Runnable interface and implements the run()  method.  After that an instance of Thread class is created by passing the instance of implemented Runnable interface.

Lets look at those two methods in detailed.

 

Creating a sub class of Thread class

Create a sub class of a Thread class and override the run() method. Then create an object of  the class and call the start() method. Then it will be executed in a separate thread.

If you check the console log, you will notice that MyThread is running in a different Thread from the main thread.

12:57:40.208 [main] INFO com.springbootdev.samples.springbootasynctask.stream.Application - Application Running on : [main]
12:57:40.223 [Thread-0] INFO com.springbootdev.samples.springbootasynctask.stream.MyThread - MyThread Running on : [Thread-0]

 

Implementing the Runnable interface

In this way, create a class that implements the Runnable interface and implement the run() method. Then create an instance of Thread class by passing the just created runnable instance.  Then call the start() method. We will modify the above example again to demonstrate how the Thread works with Runnable interface.

 

If you check the console log, you will notice that MyWork is running in a different Thread from the main thread.

12:57:40.208 [main] INFO com.springbootdev.samples.springbootasynctask.stream.Application - Application Running on : [main]
12:57:40.223 [Thread-0] INFO com.springbootdev.samples.springbootasynctask.stream.MyWork - MyWork Running on : [Thread-0]

 

What is the best way to create and run threads?  Sub class of Thread or implementation of Runnable interface?

Any of above approaches can be used to create and run Thread in java. The choice between one of them can be made depends on the personal preferences and special requires.  for instance, if you need to override the some of the default behavior of the Thread class, then you can go with extending Thread class. otherwise you can go with implementing Runnable interface.

Personally i prefer, implementing the Runnable interface and delegating the runnable instance to the Thread object.

 

Worker and Work (Work to be Done)

The relationship between Thread and Runnable can be describes as the relationship between worker and work to be done.

Runnable – Work to be done

Thread  – Worker who is handling the work (Runnable)

 

Executing the run() method directly without executing the start() method

If you just invoke run() directly, it’s executed on the calling thread, just like any other method call.

Thread.start() will create a new thread so that the runnable’s run method is executed in parallel.

when program calls start() method a new Thread is created and code inside run() method is executed in new Thread while if you call run() method directly no new Thread is created and code inside run() will execute on currently running (existing) Thread.

Another difference between start() vs run()  in Java thread is that you cannot call start() method more than one time on thread object. once started, second call of start() will throw IllegalStateException in Java while you can call run() method any number of times.

Please refer the below example.

 

 

If you check the console log, you will notice that MyThread is running in the same thread (main thread).

04:31:55.412 [main] INFO com.springbootdev.samples.springbootasynctask.stream.Application - Application Running on : [main]
04:31:55.419 [main] INFO com.springbootdev.samples.springbootasynctask.stream.MyThread - MyThread Running on : [main]

 

Creating a Thread in Java 8 with Lambda

Runnable interface is marked as a functional interface. Therefore we can use a lambda expression to create a thread as follows.

 

Execution of above program will give you the following console output.

04:37:24.334 [main] INFO com.springbootdev.samples.springbootasynctask.stream.Application - Application Running on : [main]
04:37:24.412 [Thread-0] INFO com.springbootdev.samples.springbootasynctask.stream.Application - Runnable instance running on : [Thread-0]

 

Thread.currentThread()

Thread.currentThread() will return the currently running thread object.

Thread thread = Thread.currentThread();

 

 

 

 

Java Concurrency: ScheduledExecutorService

The ScheduledExecutorService is an extended interface of ExecutorService. It is used to executes after given initial delay or execute tasks repeatedly with given intervals or both (both initial delay and repeatedly execution).  It will repeatedly execute the task until it is explicitly stopped.

 

Different thread pool implementations

  • Single Thread for handling task. (with all repeated executions)
Executors.newSingleThreadScheduledExecutor();

As the name implies this will create only a single worker thread and it will be reused for all repeated execution.

Please refer the below example.

The initial delay is 5 seconds and the interval between each execution is 2 seconds.

 

  • Thread pool for handling task (with repeated executions)
Executors.newScheduledThreadPool(5);

This will create a thread pool with five threads and those threads will be reused for all repeated execution. Each repeated execution will be handled by a randomly selected thread from the available thread pool.

Please refer the below example.

 

 

Java Concurrency: ExecutorService (A better way of asynchronous tasks handling)

The Traditional/Conventional way of handling asynchronous tasks with Thread class

When you develop a multi-threaded java application, you create a Runnable instance and corresponding Thread instance for executing them.  This is the most common and preferred way of doing concurrent programming among most of the developers. Dealing with Thread class directly may lead to few issues and those will be described below.

The issues with dealing with Thread class directly. 

The creation of new thread for each task is really expensive. This is because it involves few steps with managing the lifecycle of the Thread.  It needs to be created, started and  the entire lifecycle of the thread instance should be managed. This consumes more time and system resources. When the number of tasks are increasing/becoming high, the number of threads that need to be created will also be high.

In addition, working directly with Thread class can be error-prone. For instance, if some any error/exception occurs, it is hard to handle them with Thread class. This is because the run() method does not throw any exception. On the other hand, there is no way to check whether the thread has executed successfully because, run() method does not return anything (return type of run() method is void).

When the thread object is started, there is no way to control its behavior. Simply, there is no way to suspend its execution later. (If there is a situation to suspend the execution of some started threads, we cannot do it if we manually created and started threads). even if there are suspend() and stop() methods available in Thread class, those have been deprecated now.

The Solution

Maintaining and managing a pool of  threads that can be used to handle/execute the submitted tasks. The thread pool can be reused to handle the upcoming tasks without creating new threads for each request. This will help to reduce the number of threads created.

Find a way to throw exception and  return the output after the execution. This is possible with Callable and Future. Callable supports throwing exception from run() method and returning output with Future interface.

A Better solution with ExecutorService

ExecutorService is a higher level replacement for working with threads directly.  This is capable of executing tasks asynchronously (in background) with managed thread pool. The main advantage of ExecutorService over traditional Thread class is that,it maintains a pool of worker threads and re-use them for executing tasks (in the background). Once the execution of the task is completed, the worker thread will not go to dead state and it will come back to the pool for serving next available task. Therefore it eliminates the unnecessary cost and overhead associated with creating separate new thread for each task in traditional Thread class approach.

The ExecutorService has some useful methods which accept either Runnable or Callable instances as method parameters. The most of the methods support of returning the output that is wrapped by Future interface. Therefore the calling process(caller) can know the ultimate execution status/result of the background task.

The  ExecutorService has some methods like shutDown() and shutDownNow(). Those methods will help to control/manage the ExecutorService with already submitted and running tasks.

What are the Executor, ExecutorService and Executors?

Executor is the top level interface and ExecutorService is an extended interface of it. Executors is an utility class which has few factory methods for creating different executor services with different types of thread pools.

Here is just a class and interface level representation.

Screen Shot 2017-12-27 at 10.32.51 PM.png

Click here to view the ScheduledExecutorService article.

ThreadPoolExecutor is almost similar to the ExecutorService. 

What are the different types of executor services available?

  • ExecutorService
  • ScheduledExecutorService
  • ThreadPoolExecutor

ExecutorService with different Thread Pool implementations.

 

  • Single Thread for handling all the tasks.
   Executors.newSingleThreadExecutor();

 

Please refer the below code example for Executors.newSingleThreadExecutor();

If you run the above program, you will notice that it will create just a single thread and reuse it for executing all the submitted callable objects.

 

  • Fixed number of threads for handling all tasks
   Executors.newFixedThreadPool(5); // This will create a thread pool with 5 threads. 

 

Please refer the below code example for Executors.newFixedThreadPool(5);

If you run the above program, you will notice that there are only five threads in the thread pool. those threads will be reused to execute all the tasks. No additional threads will be created and the available five threads will be reused.

 

  • Creating new threads as needed (based on the demand) and re-using the previously constructed threads when they are available.
   Executors.newCachedThreadPool();

 

Please refer the below code example for Executors.newCachedThreadPool();

Controlling the ExecutorService

shutdown() :- The ExecutorService will not accept any new tasks once the shutdown method is called. All the previous submitted tasks will be executed before shut downing the executorService.

shutdownNow() :- The ExecutorService will be shutdown immediately by terminating already running tasks. The tasks that are pending on execution will no be executed and will be returned back as a list. So it is guaranteed that calling this method will shut down the ExecutorService immediately.

 

Either Runnable or Callable instances (Single Object or List of Objects)  will be accepted as method arguments by the execution related methods in the ExecutorService.  If you want to access the currently running thread, you can get it by Thread.currentThread() and it will return the current worker thread.

 

Java Thread: Lifecycle of Thread

The lifecycle of the Thread can be described as follows.

Screen Shot 2017-12-24 at 4.41.24 AM.png

According to the above diagram, a Thread object can be in one of the following life cycle statuses.

 

New

When the thread object is created it is in the new state.

Thread threadA = new Thread();

Now the threadA is in the new state.

 

Runnable

This is commonly known as “ready to run state“. Here the thread object is started and ready to run. There will be multiple thread objects in this state and it it the responsibility of the Thread scheduler to select a thread object for running. The thread object in new state can be moved to Runnable state by calling the start() method on it.

threadA.start();

 

Thread object can be started only once. if you try to start it more than once,  it will throw IllegalThreadStateException.

 

Running

If the Thread scheduler has selected a thread from runnable state and call the run() method, then it is in the Running state. Here the run() method is still executing and not competed yet. It is important to note that the run() method must be called by the Thread scheduler.

 

Not Runnable

This is also known as blocking, sleeping or waiting state. The currently running thread can be moved to “Not Runnable State” due to several reasons.  for instance

  • calling the sleep() method
  • calling the wait() method on the shared resource
  • calling the join() method etc….

In this case, the execution of the running thread might be hold and moved to not runnable state from running state.

 

Back to Runnable from Not Runnable state

The threads that are in the not runnable state can be moved to runnable state again on following situations.

  • sleeping/waiting time has expired
  • call the interrupt() method
  • call the notify()/notifyAll() on waited resource/object

After  it is moved back to the runnable state, it is the responsibility of the thread scheduler to select them and resume their execution.

 

Dead

Once the run() method is completed, the execution of the thread object will be over and thread will be moved to the dead state.