Documentation : Java REST Integration Tutorial

In this example we would like to show you how to easily add synchronization with mobile apps to existing system. For this example we will use Pipedrive CRM with RESTful API. Full API documentation is available here

Pipedrive provides large functionality to users and synchronizing all data does not make sence for this tutorial. We will show here how to synchronize just contacts - person and organization objects, which are in the relation. After reading this tutorial you will have known to add another entities without our help. We want to provide users posibility to list of persons and organizations, add new ones, update or delete them from mobile apps. Of course, there will be more then one user and everyone will be permit to modify all entities, because of that conflicts may occurs. We will pick one of default Mobeelizer conflict resolving strategy - overwrite - so when two users will modifiy entity in the same time, last modification will be approved. Using Mobeelizer, mobile users will have posibility to work with objects offline and when Internet access will be available they can click sync and whole modified data will appear in Pipedrive system in few moments. I have just talked about mobile users, but I haven't mentioned, who they are. I think the best idea here is to integrate Pipedrive users with Mobeelizer authentication, and we will do it. 

Before we start

Although both of these systems are using REST API we need to create additional adapter, which will be responsible for connecting them together. During creating this adapter, we need to remember that Mobeelizer Java SDK requires list of modified entities since last synchronization in sync method. Pipedrive REST API is not providing such information, so how can we get them? To resolve this problem, we will need to create separated database, which will store entities metadata needed to perform synchronization. On the following picture, there is an architecture of our example.

Design models

We have already known the architecture, and what will be synchronized. Lets create models. 

 

Create new application in App Designer called pipedrive.

 

Skip to models section and create Organization model with one text field - name.

Next create Person model. And it fields. 

  • name - TEXT
  • email - TEXT
  • phone - TEXT
  • organization - BELONGS TO Organization

That is all, models configuration is ready. The next thing to do is creating separated group of users to administrators and pipdrive users. The adapter, which we are creating, is normal Mobeelizer client and need to be authenticated with administrator rights, that is why we are creating administration group. From default there is users group, remove it first. 

Next, create new device category called pipedrive. Our adapter will be created in it. mobile device category leave as it is.

All configuration in create section is ready. We can deploy our first version to test environment. After that open test environment and skip to users tab, we will create administrator credentials to authenticate our adapter. 

First remove default user and then create new one. 

  • login: pipedrive
  • password: adapterPass

Mobeelizer cloud application is ready, download xml definition now and remember where you saved it, because we will need it later. 

Create pipedrive account

To integrate Mobeelizer with Pipdrive you have to create your own account. Don't be afraid, first 30 days are for free. You can register here. When your account is ready, it is time to get API Token. You can find it oppening your account settings and Personal section, your API Token is in API tab. 

Prepare adapter skeleton

We decided to use Spring roo in this tutorial, because we want to focus on synchronization things not creating web application in java. We will use in memory database but you can easily change it later using spring roo command line. 

Download this script and open Spring roo shell in your working directory. Next, type command, which will execute our script. 

script script.roo

After script execution, there is eclipse project created for you. There are also a few class generated in it:

  • Organization (com.mobeelizer.integration.pipedrive.model) - it is java class coresponding to Organization model which will be synchronized. There is a few additional fields in it which represens additional metadata to perform synchronization. There is also id fields which is needed to map pipdrive id with Mobeelizer guid. 
  • Person (com.mobeelizer.integration.pipedrive.model) - it is java class coresponding to Person model.
  • PipedriveUser (com.mobeelizer.integration.pipedrive.model) - java class that store information about pipdrive users. Needed for users integration.
  • OrganizationRepository (com.mobeelizer.integration.pipedrive.repository) - Organization data access object, we will need it to store organization entities in local database. 
  • PersonRepository (com.mobeelizer.integration.pipedrive.repository) - Person data access object, we will need it to store person entities in local database. 
  • UserRepository (com.mobeelizer.integration.pipedrive.repository) - User data access object, needed to store information about users to integrate with mobeelizer. 
  • OrganizationPipedriveService (com.mobeelizer.integration.pipedrive.rest.client) Pipedrive REST client, provides CRUD operation on Organization entities.
  • PersonPipedriveService (com.mobeelizer.integration.pipedrive.rest.client) Pipedrive REST client, provides CRUD operation on Person entities.
  • UserPipedriveService (com.mobeelizer.integration.pipedrive.rest.client) Pipedrive REST client, gets user entities.
  • SyncController (com.mobeelizer.integration.pipedrive.controller) - Controller that is displaying REST method to perform synchronization.
  • MobeelizerService (com.mobeelizer.integration.pipedrive.services) - Service responsible for Mobeelizer integration operations.
  • PipedriveIntegrationService (com.mobeelizer.integration.pipedrive.services) - Serivce responsible for pipedrive integration. 
  • IntegrationService (com.mobeelizer.integration.pipedrive.services) - Service responsible for controling whole synchronization process, it use MobeelizerService and PipedriveIntegrationService inside.

There are also two packages not described here, com.mobeelizer.integration.pipedrive.services.impl and com.mobeelizer.integration.pipedrive.rest.client.impl , there are class that will be implementations of defined services. 

Sync algorithm

Skeleton of application is ready, and we know what our class are for, but what is the algorithm of synchronization process. Actually it is pretty simple and it should look like this:

  1. Get modifications from Pipdrive since last synchronization. 
  2. Add new users to Mobeelizer.
  3. Synchronize local database with Mobeelizer. 
  4. Send changes to Pipdrive. 
  5. Clear database after synchronization.

As, I wrote before, responsible for controlling sync process is IntegrationService, so we can implement this algorithm in it. Open IntegrationService interface and add new method definition. 

IntegrationService.java
public interface IntegrationService {
	boolean startSynchronization();
}

 ... and implement it in IntegrationServiceImpl class. 

IntegrationServiceImpl.java
@Service
public class IntegrationServiceImpl implements IntegrationService {
	
	@Autowired
	PipedriveIntegrationService pipedriveIntegrationService;
	
	@Autowired
	MobeelizerService mobeelizerService;

	@PostConstruct
	private void initSync(){
		// TODO: implement later.
	}
	
	@Override
	public boolean startSynchronization() {
		try{
			pipedriveIntegrationService.getChanges();
			
			mobeelizerService.addNewUsers();
			
			mobeelizerService.synchronize();
			
			pipedriveIntegrationService.sendChangesToPipedrive();
			
			clearDatabase();
		}
		catch(IllegalStateException e){
			System.out.println("ERROR: While synchronization in IntegrationService: " + e.getMessage());
			return false;
		}
		return true;
	}
}

Because we are using IN MEMORY database, we will lost all local data after closing the serwer. To prevent problems caused by that we will perform some initial synchronization with Mobeelizer when serwer is starting, to retrieve lost data. For now leave initSync method empty. 

As you can see we defined our algorithm here but we have not created invoked method yet. Lets define them in our services:

MobeelizerService.java
public interface MobeelizerService {

	void synchronize();
	
	void addNewUsers();
}
PipedriveIntegrationService.java
public interface PipedriveIntegrationService {

	void getChanges();

	void sendChangesToPipedrive();
}

There is also, one private method invoked in IntegrationServiceImpl not defined yet. clearDatabase; Create it now and leave empty.

IntegrationServiceImpl.java
@Service
public class IntegrationServiceImpl implements IntegrationService {
	
	...

	private void clearDatabase() {
		// TODO: Remember to implement it.
	}
}

Ok, startSynchronization method is ready, we can display it in SyncController. 

SyncController.java
@RequestMapping("/synchronization")
@Controller
public class SyncController {
	
	@Autowired
	private IntegrationService integerationService;
	
	@RequestMapping(value = "perform")
	@ResponseBody
	public String performSync() throws JSONException{
		JSONObject response = new JSONObject();
		response.put("success", integerationService.startSynchronization());
		return response.toString();
	}
}

Get modifications from Pipedrive

We will implement our algorithm step by step, first one is get modifications from Pipdrive. In this method we need to get informations about new users and store them in local database, and get modified contacts since last synchronization. 

There is no method in Pipedrive REST API to get new users, so we are getting all of them and checking, if we already added them to Mobeelizer. 

PipedriveIngetrationService.java
 @Service
public class PipedriveIntegrationServiceImpl implements PipedriveIntegrationService {
	@Autowired
	private UserPipedriveService userPipedriveService;
	
	@Autowired
	private OrganizationPipedriveService organizationPipedriveService;
	@Autowired
	private UserRepository userRepository;
	
	@Autowired
	private OrganizationRepository organizationRepository;
	
	@Override
	public void getChanges() {
		for(PipedriveUser user : userPipedriveService.findAll()){
			PipedriveUser databaseUser = userRepository.findById(user.getId());
			if(databaseUser == null){
				user.setModified(true);
				userRepository.save(user);
			}
		}
 
		// TODO: get organizations and persons modifications


	}
}

With persons and organizations, there is more work to do, we need to find missing entities, modified and new ones. To find missing ones we will mark all entities in database as deleted, while getting Pipdrive lists we will unmark existing ones and check, if there are not changed since last synchronization. 

PipedriveIntegrationService.java
	@Override
	public void getChanges() {
			
		...
		
		organizationRepository.setDeletedForAll(true);
		
		for(Organization organization : organizationPipedriveService.findAll()){
			Organization databaseOrganization = organizationRepository.findById(organization.getId());
			if(databaseOrganization == null){
				databaseOrganization = new Organization();
				databaseOrganization.setId(organization.getId());
				databaseOrganization.setGuid(UUID.randomUUID().toString());
				databaseOrganization.setModified(true);
				databaseOrganization.setName(organization.getName());
				databaseOrganization.setOwner(organization.getOwner());
			}
			else{
				if(!databaseOrganization.getName().equals(organization.getName())){
					databaseOrganization.setName(organization.getName());
					databaseOrganization.setModified(true);
				}
			}
			databaseOrganization.setDeleted(false);
			organizationRepository.save(databaseOrganization);
		}
	
		personRepository.setDeletedForAll(true);
		
		for(Person person : personPipedriveService.findAll()){
			Person databasePerson = personRepository.findById(person.getId());
			if(databasePerson == null){
				databasePerson = new Person();
				databasePerson.setId(person.getId());
				databasePerson.setGuid(UUID.randomUUID().toString());
				databasePerson.setName(person.getName());
				databasePerson.setEmail(person.getEmail());
				databasePerson.setPhone(person.getPhone());
				databasePerson.setOrganization(person.getOrganization());
				databasePerson.setOwner(person.getOwner());
				databasePerson.setModified(true);
			}
			else {
				if(!databasePerson.getName().equals(person.getName())){
					databasePerson.setName(person.getName());
					databasePerson.setModified(true);
				}
				if(!databasePerson.getEmail().equals(person.getEmail())){
					databasePerson.setEmail(person.getEmail());
					databasePerson.setModified(true);
				}
				if(!databasePerson.getPhone().equals(person.getPhone())){
					databasePerson.setPhone(person.getPhone());
					databasePerson.setModified(true);
				}
				if(!databasePerson.getOrganization().equals(person.getOrganization())){
					databasePerson.setOrganization(person.getOrganization());
					databasePerson.setModified(true);
				}
			}
		
			databasePerson.setDeleted(false);
			personRepository.save(databasePerson);
		}
	}

It is time for implementing rest clients and repositories now. We will start with repositories.

UserRepository.java
@RooJpaRepository(domainType = PipedriveUser.class)
public interface UserRepository {

	PipedriveUser findById(int id);
}
OrganizationRepository.java
 @RooJpaRepository(domainType = Organization.class)
public interface OrganizationRepository {

	Organization findById(int id);
	
	@Modifying
	@Transactional
	@Query("update Organization o set o.deleted = ?1")
	int setDeletedForAll(boolean deleted);
	
}
PersonRepository.java
@RooJpaRepository(domainType = Person.class)
public interface PersonRepository {

	Person findById(int id);
		
	@Modifying
	@Transactional
	@Query("update Person p set p.deleted = ?1")
	int setDeletedForAll(boolean deleted);
}


We can skip to REST client now, first we have to define already invoked methods in our services. 

UserPipedriveService.java
public interface UserPipedriveService {
	
	List<PipedriveUser> findAll();	
}
OrganizationPipedriveService.java
public interface OrganizationPipedriveService {
	
	List<Organization> findAll();
}
PersonPipedriveService.java
public interface PersonPipedriveService {


	List<Person> findAll();
}

To implement this interfaces, we have to create properties file first to store our pipedrive API Token and url to pipedrive REST API. Create new file pipedrive.properties in src/main/resources/META_INT/spring/ directory and put this content in it, rememver to replace YOURS_API_TOKEN_HERE with token from your pipedrive account:

pipedrive.properties
pipedrive.url=https://api.pipedrive.com/v1/
pipedrive.api_token=YOURS_API_TOKEN_HERE

When properties are ready we can skip to UserPipedriveServiceImpl class. We will use RestTemplate class to send REST request to pipedrive, as a response type we will set String class and after success response we will parse this String to JSONObject. 

UserPipedriveServiceImpl.java
@Service
public class UserPipedriveServiceImpl implements UserPipedriveService {
	private static final String USER_LIST = "users";
	
	@Value("${pipedrive.api_token}")
	private String API_KEY; 
	@Value("${pipedrive.url}")
	private String URL;
	
	private static final String DEFAULT_PASSWORD = "password";
	
	private RestTemplate restTemplate;
	
	public UserPipedriveServiceImpl(){
		restTemplate = new RestTemplate();
	}
	
	@Override
	public List<PipedriveUser> findAll() {
		List<PipedriveUser> list = new ArrayList<PipedriveUser>();
		String url = URL+USER_LIST +"?api_token=" + API_KEY;
	
		ResponseEntity<String> obj = restTemplate.getForEntity(url, String.class);
		try {
			JSONObject json = new JSONObject(obj.getBody());
			if(json.getBoolean("success")){
				JSONArray array = json.getJSONArray("data");
				
				for(int i = 0 ; i < array.length() ; ++i){
					JSONObject entity = array.getJSONObject(i);
					PipedriveUser user = new PipedriveUser();
					user.setId(entity.getInt("id"));
					user.setName(entity.getString("email"));
					user.setDeleted(!entity.getBoolean("active_flag"));
					user.setPassword(DEFAULT_PASSWORD);
					list.add(user);
				}
			}
		} catch (JSONException e) {
			e.printStackTrace();
		}	
		return list;
	}	
}

I need to describe one think more, Pipdrive does not provide any other interface to integrate users, we can only retrieve list of them but we can't get theirs passwords. Because of that we are setting default password for mobile users here. 

The next implementations of OrganizationPipedriveService and PersonPipedriveService should be really similar to previous code. 

OrganizationPipedriveServiceImpl.java
@Service
public class OrganizationPipedriveServiceImpl implements OrganizationPipedriveService{
	private static final String ORGANIZATION_LIST = "organizations";
	
	@Value("${pipedrive.api_token}")
	public String API_KEY; 
	@Value("${pipedrive.url}")
	public String URL;
	
	@Autowired
	private UserRepository userRepository;
	
	private RestTemplate restTemplate;
	
	public OrganizationPipedriveServiceImpl(){
		restTemplate = new RestTemplate();
	}
	
	@Override
	public List<Organization> findAll() {
		List<Organization> list = new ArrayList<Organization>();
		
		String url = URL+ORGANIZATION_LIST +"?start=0&api_token=" + API_KEY; 
		
		ResponseEntity<String> obj = restTemplate.getForEntity(url, String.class);
		try {
			JSONObject json = new JSONObject(obj.getBody());
			if(json.getBoolean("success")){
				JSONArray array = json.getJSONArray("data");
				
				for(int i = 0 ; i < array.length() ; ++i){
					JSONObject entity = array.getJSONObject(i);
					Organization organization = new Organization();
					organization.setId(entity.getInt("id"));
					organization.setName(entity.getString("name"));
					organization.setOwner(entity.getJSONObject("owner_id").getString("email"));
					list.add(organization);
				}
			}
		} catch (JSONException e) {
			e.printStackTrace();
		}
		return list;
	}
}
PersonPiperdeiveServiceImpl.java
@Service
public class PersonPipedriveServiceImpl implements PersonPipedriveService{
	private static final String PERSON_LIST = "persons";
	
	@Value("${pipedrive.api_token}")
	public String API_KEY; 
	@Value("${pipedrive.url}")
	public String URL;
	
	private RestTemplate restTemplate;
	
	@Autowired
	private UserRepository userRepository;
	
	@Autowired
	private OrganizationRepository organizationRepository;
	
	public PersonPipedriveServiceImpl(){
		restTemplate = new RestTemplate();
	}
	
	@Override
	public List<Person> findAll() {
		List<Person> list = new ArrayList<Person>();
		String url = URL+PERSON_LIST +"?start=0&api_token=" + API_KEY; 
		
		ResponseEntity<String> obj = restTemplate.getForEntity(url, String.class);
		try {
			JSONObject json = new JSONObject(obj.getBody());
			if(json.getBoolean("success")){
				JSONArray array = json.getJSONArray("data");
				
				for(int i = 0 ; i < array.length() ; ++i){
					JSONObject entity = array.getJSONObject(i);
					Person person = new Person();
					person.setId(entity.getInt("id"));
					person.setName(entity.getString("name"));
					if(entity.has("email")){
						person.setEmail(((JSONObject)entity.getJSONArray("email").get(0)).getString("value"));
					}
					if(entity.has("phone")){
						person.setPhone(((JSONObject)entity.getJSONArray("phone").get(0)).getString("value"));
					}
					// Translation from pipedrive relation to Mobeelizer relation.
					// begin
					if(entity.has("org_id")){
						person.setOwner(entity.getJSONObject("owner_id").getString("email"));
					}
					int id = entity.getJSONObject("org_id").getInt("value");
					person.setOrganization(organizationRepository.findById(id).getGuid());
					// end
					list.add(person);
				}
			}
		} catch (JSONException e) {
			e.printStackTrace();
		}
		return list;
	}
}


Add users to Mobeelizer

Next step in our algorithm is adding new users to Mobeelizer. To do it we have to do some configuration first. First of all we need to add Mobeelizer Java SDK to our project. To do it open pom.xml file and add one repository and one dependency. 

pom.xml
	...
		<dependency>        
	        <groupId>com.mobeelizer</groupId>        
	        <artifactId>java-sdk</artifactId>        
	        <version>1.6.0</version>			
	    </dependency>
	...    
    	<repository>        
	   		 <id>qcadoo-releases-repository</id>        
        	<url>http://nexus.qcadoo.org/content/repositories/releases</url>
    	</repository>
	...

Next we need to add Mobeelizer application definition file into the resources. Rename downloaded on the begining of this tutorial definition XML file to application.xml and save it in src/main/resources directory. 

When application XML is ready create properties file to store properties needed to create Mobeelizer object. Create mobeelizer.properties file in src/main/resources/META-INF/spring directory.

mobeelizer.properties
mobeelizer.device=pipedrive
# Unique device id, it can be ip addres of the server or other unique value.
mobeelizer.device_id=dfgpo23rmdnsfvcx142
mobeelizer.user=pipedrive
mobeelizer.password=webAppPass
mobeelizer.pipedrive_users_group=pipedrive
mobeelizer.deffinition=/application.xml
mobeelizer.package=com.mobeelizer.integration.pipedrive.model

 Now, we can start implementing MobeelizerService. We will define post constructor method, which will create Mobeelizer object. This object will be responsible for all communication with Mobeelizer cloud. 

MobeelizerServiceImpl.java
@Service
public class MobeelizerServiceImpl implements MobeelizerService{
	
	@Value("${mobeelizer.device}")
	private String DEVICE;
	
	@Value("${mobeelizer.device_id}")
	private String DEVICE_ID;


	@Value("${mobeelizer.user}")
	private String USER;


	@Value("${mobeelizer.password}")
	private String PASSWORD;


	@Value("${mobeelizer.pipedrive_users_group}")
	private String PIPEDRIVE_USERS;


	@Value("${mobeelizer.deffinition}")
	private String DEFFINITION_FILE_PATH;
	
	@Value("${mobeelizer.package}")
	private String PACKAGE;


	@Value("${mobeelizer.push_url}")
	private String PUSH_NOTIFICATION_URL;
	
	private Mobeelizer mobeelizer;
		
	@PostConstruct
	public void init(){
		MobeelizerConfiguration configuration = new MobeelizerConfiguration();
		configuration.setDefinition(getClass().getResourceAsStream(DEFFINITION_FILE_PATH));
		configuration.setDevice(DEVICE);
		configuration.setDeviceIdentifier(DEVICE_ID);
		configuration.setPackageName(PACKAGE);
		configuration.setUser(USER);
		configuration.setPassword(PASSWORD);
		configuration.setPushNotificationUrl(PUSH_NOTIFICATION_URL);
		mobeelizer = new Mobeelizer(configuration);
	}
}

Mobeelizer object is ready, now we can implement adding new users to the Mobeelizer. 

MobeelizerServiceImpl.java
@Service
public class MobeelizerServiceImpl implements MobeelizerService{


	@Autowired
	private UserRepository userRepository;
	
	...

	@Override
	public void addNewUsers() {
		for(PipedriveUser newUser : userRepository.findByModified(true)){
			MobeelizerUser user = new MobeelizerUser();
			user.setAdmin(false);
			user.setGroup(PIPEDRIVE_USERS);
			user.setLogin(newUser.getName());
			user.setMail(newUser.getName());
			user.setPassword(newUser.getPassword());
			mobeelizer.createUser(user);
		}
	}
}

As you remember, during getting new users from Pipdrive, we were marking them with modified flag. Then they were not included in database. To get them from database now we just have to find all modified users. Rest of the code is mapping PipedriveUser object to MobeelizerUser object and invoking createUser from Mobeelizer object. 

Last thing to do is adding findByModified method to UserRepository.

UserRepository.java
@RooJpaRepository(domainType = PipedriveUser.class)
public interface UserRepository {
	...


	PipedriveUser findByModified(int id);
}


Synchronize local database with  Mobeelizer

It is time for synchronization with Mobeelizer cloud. In fact, it is just invoking sync method from Mobeelizer object. Before that we need to get differential data from local database and after synchronization has finished, we have to handle new data from cloud. 

Mobeelizer sync method gets as a parameter callback to handle synchronization data, in our tutorial we will use MobeelizerServiceImpl as a callback, because of that we need to implement second interface on it. Everything  should look like that. 

MobeelizerServiceImpl.java
@Service
public class MobeelizerServiceImpl implements MobeelizerService, MobeelizerSyncCallback{
	
	...

	@Autowired
	private OrganizationRepository organizationRepository;
	
	@Override
	public void synchronize() {
		List<Object> entities = getEntitiesToSync();
		mobeelizer.syncAndWait(entities, null, this);
	}
 
	@Override
	public void onSyncFinishedWithSuccess(Iterable<Object> entities,
			Iterable<MobeelizerFile> files, Iterable<String> deletedFiles,
			MobeelizerConfirmSyncCallback confirmCallback) {
		removeModificationFlag();
		removeDeletedEntities();
		
		applayChanges(entities);
		confirmCallback.confirm();
	}

	@Override
	public void onSyncFinishedWithError(MobeelizerOperationError error) {
		throw new IllegalStateException(error.getMessage());
	}
	@Override
	public void onSyncFinishedWithError(MobeelizerErrors databaseError) {
		StringBuilder builder = new StringBuilder("Unable to synchronize with mobeelizer. ");
		for(MobeelizerError error : databaseError.getGlobalErrors()){
			builder.append(error.getMessage());
		}
		throw new IllegalStateException(builder.toString());
	}
 
	private List<Object> getEntitiesToSync() {
		List<Object> entities = new ArrayList<Object>();
		for(Organization organizationToSync : organizationRepository.findByModifiedOrDeleted(true, true)){
			entities.add(organizationToSync);
		}
		for(Person personToSync : personRepository.findByModifiedOrDeleted(true, true)){
			entities.add(personToSync);
		}
		return entities;
	}
 
	private void removeModificationFlag() {
		organizationRepository.setModificationFlag(true, false);
		personRepository.setModificationFlag(true, false);
	}
 
	private void removeDeletedEntities() {
		organizationRepository.removeDeleted();
		personRepository.removeDeleted();
	}
 
	private void applayChanges(Iterable<Object> entities) {
		for(Object entity : entities){
			if(entity instanceof Organization){
				Organization syncedEntity = (Organization) entity;
				Organization databaseEntity = organizationRepository.findByGuid(syncedEntity.getGuid());
				if(databaseEntity == null){
					syncedEntity.setId(-1);
					syncedEntity.setModified(true);
					organizationRepository.save(syncedEntity);
				}
				else{
					databaseEntity.setModified(true);
					databaseEntity.setName(syncedEntity.getName());
					databaseEntity.setOwner(syncedEntity.getOwner());
					databaseEntity.setDeleted(syncedEntity.isDeleted());
					organizationRepository.save(databaseEntity);
				}
			}
			if(entity instanceof Person){
				Person personFromCloud = (Person) entity;
				Person databaseEntity = personRepository.findByGuid(personFromCloud.getGuid());
				if(databaseEntity == null){
					personFromCloud.setId(-1);
					personFromCloud.setModified(true);
					personRepository.save(personFromCloud);
				}
				else{
					databaseEntity.setModified(true);
					databaseEntity.setName(personFromCloud.getName());
					databaseEntity.setEmail(personFromCloud.getEmail());
					databaseEntity.setPhone(personFromCloud.getPhone());
					databaseEntity.setOrganization(personFromCloud.getOrganization());
					databaseEntity.setDeleted(personFromCloud.isDeleted());
					personRepository.save(databaseEntity);
				}
			}
		}
	}
}

There are lot of methods here. Lets describe them step by step. We are starting in synchroize() method. We want to invoke syncAndWait method from Mobeelizer object. To do that we have to get modified entities. There is getEntitiesToSync method designed for that. In this method, we are getting whole modified and deleted entities and then returning them like a list. After invoking Mobeelizer syncAndWait method, Mobeelizer SDK will invoke one of methods: onSyncFinishedWithSuccess or onSyncFinishedWithError. When synchronization has finished with success we are removing modification flag and cleaning deleted entities, because there are already synchronized with success with the cloud. Next we are applying changes from the cloud and confirming synchronization. In applayChnges method we are adding new entities from the cloud to local database and modyfing existing ones. We are again marking them with modified flag to store information which entities we have to send to pipedrive. 

Last thing to do here is creating all required methods in repositories class.

OrganizationRepository.java
@RooJpaRepository(domainType = Organization.class)
public interface OrganizationRepository {


	...	
 
	Iterable<Organization> findByModifiedOrDeleted(boolean modified, boolean deleted);
	
	Organization findByGuid(String guid);
	 
	@Modifying
	@Transactional
	@Query("update Organization o set o.modified = ?2 where o.modified = ?1")
	int setModificationFlag(boolean from, boolean to);
	
	@Modifying
	@Transactional
	@Query("delete from Organization u where u.deleted = true")
	void removeDeleted();
	
}

PersonRepository.java
@RooJpaRepository(domainType = Person.class)
public interface PersonRepository {
	...
 
	Iterable<Person> findByModifiedOrDeleted(boolean modified, boolean deleted);
 
	Person findByGuid(String guid);

	@Modifying
	@Transactional
	@Query("update Person p set p.modified = ?2 where p.modified = ?1")
	int setModificationFlag(boolean from, boolean to);

	@Modifying
	@Transactional
	@Query("delete from Person p where p.deleted = true")
	void removeDeleted();
}

Send changes to the Pipedrive

Last, but not the least thing to do is sending changes to Pipedrive. In this method, we will find persons or organizations to remove, modify and add. Then we will invoke method from our Rest Clients. 

PipedriveIntegrationServiceImpl.java
@Service
public class PipedriveIntegrationServiceImpl implements PipedriveIntegrationService {


	...
 
	@Override
	public void sendChangesToPipedrive() {
		for(Organization organization : organizationRepository.findByDeleted(true)){
			organizationPipedriveService.delete(organization);
		}
		
		for(Organization organization : organizationRepository.findByModifiedAndId(true, -1)){
			int id = organizationPipedriveService.insert(organization);
			organization.setId(id);
			organization.setModified(false);
			organizationRepository.save(organization);
		}
		
		for(Organization organization : organizationRepository.findByModified(true)){
			organizationPipedriveService.update(organization);
			organization.setModified(false);
			organizationRepository.save(organization);
		}	
		
		for(Person person : personRepository.findByDeleted(true)){
			personPipedriveService.delete(person);
			person.setModified(false);
			personRepository.save(person);
		}
		
		for(Person person : personRepository.findByModifiedAndId(true, -1)){
			int id = personPipedriveService.insert(person);
			person.setId(id);
			person.setModified(false);
			personRepository.save(person);
		}
		
		for(Person person : personRepository.findByModified(true)){
			personPipedriveService.update(person);
			person.setModified(false);
			personRepository.save(person);
		}
	}
}

When entity id is set to -1 we know that it was created on mobile, we have to invoke insert method and then update ID in local database. When ID already was in database, it means that entity was just modified. 

To get all this information from database we have to create following repositories methods.

OrganizationRepository.java
@RooJpaRepository(domainType = Organization.class)
public interface OrganizationRepository {
 
	Iterable<Organization> findByDeleted(boolean deleted);
 
	Iterable<Organization> findByModifiedAndId(boolean modified, int id);

	Iterable<Organization> findByModified(boolean modified);
}
PersonRepository.java
@RooJpaRepository(domainType = Person.class)
public interface PersonRepository {
	...
 
	Iterable<Person> findByDeleted(boolean deleted);

	Iterable<Person> findByModifiedAndId(boolean modified, int id);

	Iterable<Person> findByModified(boolean modified); 
}

 

Next thing to do is creating rest clients methods. 

OrganizationPipedriveService.java
public interface OrganizationPipedriveService {
	...
	
	void update(Organization organization);
	
	int insert(Organization organization);
	
	void delete(Organization organization);
}
PersonPipedriveService.java
public interface PersonPipedriveService {
	...

	int insert(Person person);

	void update(Person person);

	void delete(Person person);
}

.. and implementing them. 

OrganizationPipedriveServiceImpl.java
@Service
public class OrganizationPipedriveServiceImpl implements OrganizationPipedriveService{
	...
 
	@Override
	public void update(Organization organization) {
		String url = URL+ ORGANIZATION_LIST +"/"+organization.getId()+"?api_token=" + API_KEY;
		JSONObject request = new JSONObject(); 
		try {
			request.put("name", organization.getName());
			request.put("owner_id", userRepository.findByName(organization.getOwner()).getId());
		} catch (JSONException e) {
			e.printStackTrace();
			throw new IllegalStateException("Error while creating request", e);
		}
		HttpHeaders headers = new HttpHeaders();
		headers.setContentType(MediaType.APPLICATION_JSON);
		HttpEntity<String> entity = new HttpEntity<String>(request.toString(),headers);
		restTemplate.put(url, entity);
	}
 
	@Override
	public int insert(Organization organization) {
		String url = URL+ORGANIZATION_LIST +"?api_token=" + API_KEY;
		
		JSONObject request = new JSONObject(); 
		try {
			request.put("name", organization.getName());
			request.put("owner_id", userRepository.findByName(organization.getOwner()).getId());
		} catch (JSONException e) {
			e.printStackTrace();
			throw new IllegalStateException("Error while creating request", e);
		}
		HttpHeaders headers = new HttpHeaders();
		headers.setContentType(MediaType.APPLICATION_JSON);
		HttpEntity<String> entity = new HttpEntity<String>(request.toString(),headers);
		ResponseEntity<String> obj = restTemplate.postForEntity(url, entity, String.class);
		try {
			JSONObject json = new JSONObject(obj.getBody());
			if(json.getBoolean("success")){
				return json.getJSONObject("data").getInt("id");	
			} 
			else{
				throw new IllegalStateException("Unable to add organization");
			}
		} catch (JSONException e) {
			e.printStackTrace();
			throw new IllegalStateException("Error while parsing response", e);
		}
	}

	@Override
	public void delete(Organization organization) {
		String url = URL+ORGANIZATION_LIST +"/"+organization.getId()+"?api_token=" + API_KEY;
		try{
			restTemplate.delete(url);
		}
		catch(HttpClientErrorException e){
			System.out.println(e.getStatusText());
			
		}
	}
}

PersonPipedriveServiceImpl.java
@Service
public class PersonPipedriveServiceImpl implements PersonPipedriveService{
	...

	@Override
	public int insert(Person person) {
		String url = URL+PERSON_LIST +"?api_token=" + API_KEY;
		
		JSONObject request = new JSONObject(); 
		try {
			request.put("name", person.getName());
			request.put("email", person.getEmail());
			request.put("phone", person.getPhone());
			request.put("org_id", organizationRepository.findByGuid(person.getOrganization()).getId());
			request.put("owner_id", userRepository.findByName(person.getOwner()).getId());
		} catch (JSONException e) {
			e.printStackTrace();
			throw new IllegalStateException("Error while creating request", e);
		}
		HttpHeaders headers = new HttpHeaders();
		headers.setContentType(MediaType.APPLICATION_JSON);
		HttpEntity<String> entity = new HttpEntity<String>(request.toString(),headers);
		ResponseEntity<String> obj = restTemplate.postForEntity(url, entity, String.class);
		try {
			JSONObject json = new JSONObject(obj.getBody());
			if(json.getBoolean("success")){
				return json.getJSONObject("data").getInt("id");	
			} 
			else{
				throw new IllegalStateException("Unable to add organization");
			}
		} catch (JSONException e) {
			e.printStackTrace();
			throw new IllegalStateException("Error while parsing response", e);
		}
	}
	@Override
	public void update(Person person) {
		String url = URL+PERSON_LIST +"/"+person.getId()+"?api_token=" + API_KEY;
		JSONObject request = new JSONObject(); 
		try {
			request.put("name", person.getName());
			request.put("email", person.getEmail());
			request.put("phone", person.getPhone());
			request.put("org_id", organizationRepository.findByGuid(person.getOrganization()).getId());
			request.put("owner_id", userRepository.findByName(person.getOwner()).getId());
		} catch (JSONException e) {
			e.printStackTrace();
			throw new IllegalStateException("Error while creating request", e);
		}
		HttpHeaders headers = new HttpHeaders();
		headers.setContentType(MediaType.APPLICATION_JSON);
		HttpEntity<String> entity = new HttpEntity<String>(request.toString(),headers);
		restTemplate.put(url, entity);
	}
	@Override
	public void delete(Person person) {
		if(person.getId() == -1){
			return;
		}
		
		String url = URL+PERSON_LIST +"/"+person.getId()+"?api_token=" + API_KEY;
		System.out.println(url);
		try{
			restTemplate.delete(url);
		}
		catch(HttpClientErrorException e){
			System.out.println(e.getStatusText());
			
		}	
	}
}

In this methods we are using RestTemplate to invoke REST Pipedrive methods described here.

Clear database after sync

Last thing to do is removing deleted entities and modifications flag from all entities. We created method for that at the begining in IntegrationServiceImpl class. We can implement it now. 

IntegrtionServiceImpl.java
@Service
public class IntegrationServiceImpl implements IntegrationService {
	
	@Autowired
	private UserRepository userRepository;
	
	@Autowired
	private OrganizationRepository organizationRepository;
	
	...

	private void clearDatabase() {
		organizationRepository.removeDeleted();
		personRepository.removeDeleted();
		organizationRepository.setModificationFlag(true, false);
		personRepository.setModificationFlag(true, false);
		userRepository.setModificationFlag(true, false);
	}
}

There is no setModificationFlag in UserRepository yet, so lets create it. 

UserRepository.java
@RooJpaRepository(domainType = PipedriveUser.class)
public interface UserRepository {
 
	...
 
	@Modifying
	@Transactional
	@Query("update PipedriveUser o set o.modified = ?2 where o.modified = ?1")
	int setModificationFlag(boolean from, boolean to);
}

Our adapter is completed now, if you did not test your code before it is greate time to do that. To invoke sync method run your service on web container, you can type in your console this command:

mvn jetty:run

After server is up, type this url addres in your web browser: 

http://localhost:8080/pipedrive/synchronization/perform

To check if data is available on the cloud, you can use Mobeelize Data Browser. You will find it in App Designer test environment in Data Browser tab. 

Initial synchronization

If you tested your code before, there would be some data on the cloud and algorithm would not work correctly after restarting server. To resolve this issue we have already defined initSync method in IntegerationServiceImpl class. In this method, we will get all data, which is already on the cloud and then update entities id from pipedrive. 

IntegrationServiceImpl.java
@Service
public class IntegrationServiceImpl implements IntegrationService {	
	...
 
	@PostConstruct
	private void initSync(){
		if(mobeelizerService.isInitSycnRequired()){
			mobeelizerService.initialSync();
			
			pipedriveIntegrationService.updateIds();
			
			clearDatabase();
		}
	}
	...
}
MobeelizerService.java
public interface MobeelizerService {
	...
 
	void initialSync();
	
	boolean isInitSycnRequired();
}
MobeelizerServiceImpl
@Service
public class MobeelizerServiceImpl implements MobeelizerService, MobeelizerSyncCallback{
	...

	@Override
	public void initialSync(){
		getAllUsersFromMobeelizer();
		
		mobeelizer.syncAllAndWait(this);
	}

	// When users table is empty that means there is init sync required.
	@Override
	public boolean isInitSycnRequired() { 
		List<PipedriveUser> users = userRepository.findAll();
		return users == null || users.size() == 0;
	}
 
	private void getAllUsersFromMobeelizer() {
		List<MobeelizerUser> users = mobeelizer.getUsers();
		for(MobeelizerUser user : users){
			if(user.getGroup().equals(PIPEDRIVE_USERS)){
				PipedriveUser newUser = new PipedriveUser();
				newUser.setId(-1);
				newUser.setName(user.getLogin());
				newUser.setPassword(user.getPassword());
				System.out.println("Saving user from Mobeelizer :" + user.getLogin());
				userRepository.save(newUser);
			}
		}
	}
}
PipedriveIntegrationService.java
public interface PipedriveIntegrationService {
	...
 
	void updateIds();
}
PipedriveIntegrationServiceImpl.java
@Service
public class PipedriveIntegrationServiceImpl implements PipedriveIntegrationService {


	...
 
	@Override
	public void updateIds() {
		for(PipedriveUser user : userPipedriveService.findAll()){
			PipedriveUser databaseUser = userRepository.findByNameAndId(user.getName(), -1);
			if(databaseUser != null){
				databaseUser.setId(user.getId());
				userRepository.save(databaseUser);
			}
		}
		
		for(Organization organization : organizationPipedriveService.findAll()){
			Organization databaseOrganization = organizationRepository.findByNameAndId(organization.getName(), -1);
			if(databaseOrganization != null){
				databaseOrganization.setId(organization.getId());
				organizationRepository.save(databaseOrganization);
			}
		}
		for(Person person : personPipedriveService.findAll()){
			Person databasePerson = personRepository.findByEmailAndId(person.getEmail(), -1);
			if(databasePerson != null){
				databasePerson.setId(person.getId());
				personRepository.save(databasePerson);
			}
		}
	}
}

Conclusion


Our adapter is ready, you can create mobile application now to finish this tutorial. 

In this part we learned how to integrate Mobeelizer with existing system. We used RESTFull API to get entities from Pipedrive, but there is no problem to get them directly from database or use another API in your application. We also have to get all entities from database before each synchronization, but in your application you can build some notification mechanism that will give you oportunity to update metadata after each modification operation.


As you can see there is no need to modify you database structure to integrate with Mobeelizer. You can create separated database to store additional metadata required to perform synchronization. This solution also protect you from vendor lock-in problem. Mobeelizer in this case is just another layer between your web app and mobile apps, which you can change if you want.


Attachments:

BeFunky_SavedPicture.png (image/png)
pipedrive_app.png (image/png)
organization_model.png (image/png)
name_field.png (image/png)
administration_group.png (image/png)
pipedrive_group.png (image/png)
pipedrive_group.png (image/png)
pipedrive_user.png (image/png)
personModel.png (image/png)
person_name.png (image/png)
perdon_email.png (image/png)
person_phone.png (image/png)
person_organization.png (image/png)
script.roo (application/octet-stream)