Java with a passion!
Java7
Copying DynamoDB Table Contents Across Regions
May 20th
There is no direct way of copying data of a DynamoDB table across regions (i.e. from one region to another) from the DynamoDB management console.
Why would you need such an operation? i.e. moving data across regions. Answer may be, sometimes you need to setup multiple environments, one for each development phase. Like for example, development, SIT, UAT, Pre-production, performance-test environment etc.
I will be deploying a solution which requires using the Amazon EMR (Elastic Map Reduce). EMR is essentially the service recommended by Amazon to move data in and out of DynamoDB into S3 where it may then be picked up for further processing / use.
High Level Overview of the solution requiring EMR
- Start an EMR Job from the EMR Management Console, putty into the EMR Master Node, start Hive.
- Set the DynamoDB endpoint to the source region
- Set up a Hive table to reference to the DynamoDB table in the source region
- Create an S3 Bucket to store temporal data
- Set up a Hive table to reference to the S3 location where data from the DynamoDB table in the source region will be written to
- Issue the Hive command that does the actual copying from the DynamoDB table in the source region to the S3 location
- Set the DynamoDB endpoint to the destination region.
- Issue the Hive command that does the actual copying from the S3 location to the DynamoDB table in the destination region.
Step 1: Start an EMR Job from the EMR Management Console, putty into the EMR Master Console, start Hive.
This step is well detailed in Amazon EMR’s Documentation here at http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/EMRforDynamoDB.html therefore i will not reiterate.
Follow the following steps in sequence
Step 1.1 (Create a Key pair): http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/EMR_SetUp_KeyPair.html
Step 1.2 (Create a Cluster): http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/EMR_CreateJobFlow.html
Step 1.3 (SSH into the Master Node): http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/EMR_SetUp_SSH.html
Once the above three steps are completed, you would have secure shelled into the EMR Master Node and started Hive.
For example:
Step 2: Set the DynamoDB endpoint to the source region
Now you will need to set the DynamoDB endpoint to the source region, this is the region where your source table resides. In this example, i am setting the source region endpoint to Singapore.
SET dynamodb.endpoint=dynamodb.ap-southeast-1.amazonaws.com;
For example:
Step 3: Set up a Hive table to reference to the DynamoDB table in the source region
In the earlier step i have pointed Hive to DynamoDB in the source region, i shall now create an external table in Hive to reference to the DynamoDB table.
CREATE EXTERNAL TABLE hive_dynamodb_customer_lines (cust_id string, lines bigint)
STORED BY 'org.apache.hadoop.hive.dynamodb.DynamoDBStorageHandler'
TBLPROPERTIES ("dynamodb.table.name" = "customer_lines",
"dynamodb.column.mapping" = "cust_id:cust_id,lines:lines");
In this script i am creating an external table named ‘hive_dynamodb_customer_lines’, this Hive table would reference the DynamoDB table ‘customer_lines’ containing the actual data. The table ‘customer_lines’ has the fields cust_id and lines.
Step 4: Create an S3 Bucket to store temporal data
In this step i am creating an S3 Bucket to store temporal data. This bucket and its contents can be deleted at the end of the data migration.
Create a bucket in S3, in this example, i am have created the bucket named ‘dynamodb-migration’. This can be named anything for you, subject to availability (i.e. there cannot be two buckets with the same time in S3)
For example:
In the newly created S3 Bucket, create a folder named ‘temp’.
For example:
Step 5: Set up a Hive table to reference to the S3 location where data from the DynamoDB table in the source region will be written to
I now need to create a Hive table that references to the newly created S3 location where i would dump the data of the DynamoDB table into.
CREATE EXTERNAL TABLE hive_s3_customer_lines (cust_id string, lines bigint) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' LOCATION 's3://dynamodb-migration/temp';
Step 6: Issue the Hive command that does the actual copying from the DynamoDB table in the source region to the S3 location
In this step i will issue the Hive command that does the copying from the DynamoDB table to S3.
INSERT OVERWRITE TABLE hive_s3_customer_lines SELECT * FROM hive_dynamodb_customer_lines;
Once this step executes, data from the DynamoDB table will be copied over into S3. You could go into your S3 bucket to see this file.
For example:
Step 7: Set the DynamoDB endpoint to the destination region.
In this step i am setting the DynamoDB endpoint to the destination region where i want the contents of the DynamoDB table to be copied into. In this example, my destination DynamoDB region would be Ireland.
SET dynamodb.endpoint=dynamodb.eu-west-1.amazonaws.com;
Step 8: Issue the Hive command that does the actual copying from the S3 location to the DynamoDB table in the destination region.
In this step i am issuing the hive command that performs the actual copy from S3 to the DynamoDB table in the destination region.
INSERT OVERWRITE TABLE hive_dynamodb_customer_lines SELECT * FROM hive_s3_customer_lines;
When completed, you could go to the DynamoDB table in the destination region to verify that your data is copied successfully.
Remember to terminate the EMR Job created through Step 1 and delete the S3 Bucket created in Step 4 to stop incurring AWS resources / charges at the end of the exercise.
Graceful URL Redirection with Amazon S3
May 14th
We have two web applications running on AWS Beanstalk which is a Tomcat container PAAS (without Apache HTTPD) and running into the classic problem of being unable to redirect URLs out of the box. Fortunately, graceful workaround exists.
Background of the applications
The buy flow for a typical customer consists of two steps, first by reserving, and then purchasing once the stock has arrived. To do so, they use the Reserve and Purchase web applications respectively.
Each of these applications supports multiple handsets on one web application. I.e. there could be 100 handsets, but only one Reserve and Purchase web application. To do this, each application differentiates one product from another using a GET parameter. E.g. model=iphone5, model=s4, model=1x. You will see more of this later.
Web Application 1 (Reserve Web Application):
The Reserve web application captures reservation information for a handset product which I wish to market, for example it could be Samsung S4, IPhone 5 or HTC One X.
The Reserve web application has the URL reserve.elasticbeanstalk.com. (Fictitious, for illustration only)
Customers who wish to reserve a handset would have to fill up and submit a form using the application’s URL reserve.elasticbeanstalk.com.
For example, for Samsung S4, the URL would be reserve.elasticbeanstalk.com/index.xhtml?model=s4.
If the handset to reserve were to be an IPhone 5, then this URL becomes reserve.elasticbeanstalk.com/index.xhtml?model=iphone5. If you notice correctly, only the value for the GET parameter ‘model’ changes.
Web Application 2 (Purchase Web Application):
The Purchase web application is the second web application that captures order information of a handset product which I wish to sell to my customers.
The Purchase web application has the URL purchase.elasticbeanstalk.com. (Fictitious, for illustration only)
Customers who wish to purchase a handset after reserving would have to fill in payment information through a form using the application’s URL purchase.elasticbeanstalk.com.
For example, for Samsung S4, the URL would be purchase.elasticbeanstalk.com/index.xhtml?model=s4.
If the handset to purchase were to be an IPhone 5, then this URL becomes purchase.elasticbeanstalk.com/index.xhtml?model=iphone5. If you notice correctly, only the value for the parameter ‘model’ changes, analogous to how the Reserve web application works.
Accessing the Web Applications
To reserve a handset, customers could use the Reserve web application’s URL indicating the model, for example, purchase.elasticbeanstalk.com/index.xhtml?model=iphone5. To purchase, customers could use the Purchase web application’s URL in a similar fashion, for example, reserve.elasticbeanstalk.com/index.xhtml?model=iphone5.
Sub Domain launch.java.sg
We have a sub domain launch.java.sg specially elected to cater to this service, this URL would serve as a single access point to customers who wish to purchase and reserve handsets.
For example, if customers were to reserve an IPhone5, then launch.java.sg/reserveIPhone5 would be used. If customers were to purchase an IPhone5, then launch.java.sg/purchaseIPhone5 would be used.
Therefore, we need some kind of redirection for reservations to go to Reserve web application’s URL and purchase to go to Purchase web application’s URL, together with the GET parameter to indicate the model.
Step 1: Create an S3 bucket representing your sub domain URL
This step requires you to go into S3 to create a bucket corresponding to the sub domain, for our case it was ‘launch.java.sg’.
Create two files named “reserveIPhone5″ and “purchaseIPhone5″, upload to the newly created S3 bucket.
Select the object “reserveIPhone5″, in the properties tab, grant an Open/Download permission to everyone. Save when completed.
Do the same for the “purchaseIPhone5″ object.
Select the “reserveIPhone5″ object again, in the metadata tab, add a Website Redirect Location metadata. In the value, enter the URL of Reserve Web Application to reserve an IPhone 5: reserve.elasticbeanstalk.com/index.jsf?index.xhtml?model=iphone5. Save when completed.
Select the “purchaseIPhone5″ object again, in the metadata tab, add a Website Redirect Location metadata. In the value, enter the URL of Purchase Web Application to purchase an IPhone 5: purchase.elasticbeanstalk.com/index.jsf?index.xhtml?model=iphone5. Save when completed.
Go back to the S3 bucket “launch.java.sg”, on the Static Website Hosting tab, select “Enable Website Hosting”. Enter a value for the index document. Save when completed.
In the Permissions tab, grant everyone the permission to list. Save when completed.
In the Static Website Hosting tab, take note of the endpoint URL. This is required to create a CNAME in the later steps.
Step 2: Create a CNAME to the S3 bucket
For me, i am using the CPANEL to add a CNAME to my domain name.
Name: launch.java.sg.
TTL: 14400
Type: CNAME
CNAME: launch.java.sg.s3-website-ap-southeast-1.amazonaws.com
Step 3: Test
Open a web browser, enter the URL http://launch.java.sg/reserveIPhone5. If everything is successful, you should be redirected to reserve.elasticbeanstalk.com/index.jsf?index.xhtml?model=iphone5
After that, enter the URL http://launch.java.sg/purchaseIPhone5. This should be redirected to purchase.elasticbeanstalk.com/index.jsf?index.xhtml?model=iphone5
:)
Java Class Version Number (48, 49, 50, 51 etc) Quick Reference
Apr 11th
Often when you receive some Java Jar files you may come across incompatibility issues when running that on your JRE.
The run-time environment must be the same or higher than the Java version that the Java source codes were compiled using
For example if you have compiled some Java source files using JDK 6, then you need at least a JDK 6 and above run-time environment to execute the Java classes.
Common Java class version incompatibility error message:
(I was trying to run this Java 7 compiled class in JRE 6.
Caused by: java.lang.UnsupportedClassVersionError: com/bim/entityfacade/rela/espagecontent/dao/PageContentEntityFacadeDAO : Unsupported major.minor version 51.0 (unable to load class com.bim.entityfacade.rela.espagecontent.dao.PageContentEntityFacadeDAO)
So whats 51.0? That is Java 7, meaning that the class file was compiled using JDK 7.
Here is the full list:
46 = Java 1.2
47 = Java 1.3
48 = Java 1.4
49 = Java 5
50 = Java 6
51 = Java 7
? = Java 8
JSF 2.0 Custom Validator Calling @EJB Injected Bean
Dec 31st
One of the wonders of JSF 2.0 that i love is the simplicity of creating a custom validator for another mere business scenario validation.
The highlights of this post showcase:
- Creating custom validator in JSF 2.0
- How it is possible to call on injected @EJB bean from the Validator
Scenario
Consider the business scenario where a form collects a price model name from the user. The business rule is that the price model name must be unique, i.e. it has not been used before.
Validation error message showing that the name is already in use.
To determine whether my price model name has been used, the application needs to check the DBMS if the price model name has already been used. Now, you could implement the logic of creating a connection to the DBMS, performing the if\else check all in the Validator. But i hope this is not what you are going to do in any real scenario because it violates many principles. This tutorial will show you how you can call on an injected @EJB bean that would contain all the business logic to check if a price model name is in use.
JSF 2.0 Validator
JSF 2.0 has a Validator framework that allows you to write a custom validation in a class, and then tie this validation class to a XHTML boundary page.
package sg.java.hms.validator.manage.pricing;
import javax.ejb.EJB;
import javax.faces.application.FacesMessage;
import javax.faces.bean.ManagedBean;
import javax.faces.bean.RequestScoped;
import javax.faces.component.UIComponent;
import javax.faces.context.FacesContext;
import javax.faces.validator.Validator;
import javax.faces.validator.ValidatorException;
import sg.java.hms.service.manage.pricing.IPriceModelService;
/**
* JSF Custom Validator.
* <p>
* Validates that the price model name is not duplicate.
* </p>
*
* @author Chin Boon
*
*/
@ManagedBean
@RequestScoped
public class PriceModelNameValidator implements Validator {
@EJB
private IPriceModelService priceModelService;
@Override
public void validate(FacesContext arg0, UIComponent arg1, Object value)
throws ValidatorException {
// cast value into String
final String priceModelName = value.toString();
// call business method to check if price model name exists
boolean priceModelNameExists = this.priceModelService
.checkPriceModelNameExists(priceModelName);
// if price model name exists, fail the validation
if (priceModelNameExists) {
FacesMessage msg = new FacesMessage(priceModelName
+ " is already in use. Please choose another name");
msg.setSeverity(FacesMessage.SEVERITY_ERROR);
throw new ValidatorException(msg);
}
}
}
You may have noticed that i am not decorating my validator class with @FacesValidator. That is because i am going to call an @EJB bean from the validator and because @FacesValidator isn’t managed by the injecton container (Answered by BalusC), i would be unable to call the injected @EJB bean that i wanted had i used @FacesValidator.
PriceModelService.java
Lets assume that priceModelService is an EJB3 bean and the method checkPriceModelNameExists() performs all the work of communicating with the DBMS and returns the result true if a price model name exists and false otherwise.
createPriceModel.xhtml
After writing up the custom validator class, binding the validator to the input field is sufficient to make the new validation take effect!
<p:inputText id="priceModelName" size="30"
value="#{priceModelBackingBean.priceModel.priceModelName}"
required="true" label="#{labelResource.priceModelName}">
<f:validator binding="#{priceModelNameValidator}" />
</p:inputText>
Conclusion
The simplicity and cleanliness of the JSF 2.0 Validator framework to provide extensive business validation scenario that would otherwise had been troublesome to create and maintain makes JSF 2.0 a very developer friendly framework to work with.
Target Unreachable, xxx returned null
Dec 22nd
This error is probably happening when you are trying to submit a form. JSF tries to populate all the user entered form values into the corresponding backing bean attributes that aren’t yet initialized, i.e. null.
Consider this simple form where we are trying to collect the Price Model Name from the user.
createPriceModel.xhtml JSF2 Form
In the form, the value of the price model name text field is set into a backing bean containing a complex data object PriceModel which contains the attribute priceModelName.
<h:form>
<p:inputText id="priceModelName"
value="#{priceModelBackingBean.priceModel.priceModelName}" />
</p:inputText>
...
</h:form>
PriceModelBackingBean.java Backing Bean
@ManagedBean
@SessionScoped
public class PriceModelBackingBean implements IPriceModelBackingBean {
private PriceModel priceModel;
...
accessors and mutators
...
}
PriceModel.java POJO
public class PriceModel {
private String priceModelName;
...
accessors and mutators
...
}
So, what could go wrong?
Solution
Initialize the priceModel attribute!
You did not initialize the value of priceModel in the backing bean. When the user submits the form and JSF tries to populate the value of the price model name which is “Standard Price Model”, it hits a null pointer exception.
While we could do this in a constructor of the backing bean, it is not advisable to do so, instead use the @PostConstruct JSF bean lifecycle annotation to help you. This way it is guaranteed that the attribute priceModel is initialized.
public class PriceModelBackingBean implements IPriceModelBackingBean {
private PriceModel priceModel;
@PostConstruct
public void init() {
this.priceModel = new PriceModel();
}
...
}
Error Message Encountered (Reference)
22:05:11,706 WARNING [javax.enterprise.resource.webcontainer.jsf.lifecycle] (http-localhost-127.0.0.1-8080-1) /facelet/manage/pricing/createPriceModel.xhtml @47,64 value="#{priceModelBackingBean.priceModel.priceModelName}": Target Unreachable, 'priceModel' returned null: javax.el.PropertyNotFoundException: /facelet/manage/pricing/createPriceModel.xhtml @47,64 value="#{priceModelBackingBean.priceModel.priceModelName}": Target Unreachable, 'priceModel' returned null
at com.sun.faces.facelets.el.TagValueExpression.getType(TagValueExpression.java:100) [jsf-impl-2.1.7-jbossorg-2.jar:]
at org.primefaces.renderkit.InputRenderer.findImplicitConverter(InputRenderer.java:170) [primefaces-3.4.jar:]
at org.primefaces.renderkit.InputRenderer.findConverter(InputRenderer.java:190) [primefaces-3.4.jar:]
at org.primefaces.renderkit.InputRenderer.getConvertedValue(InputRenderer.java:196) [primefaces-3.4.jar:]
at javax.faces.component.UIInput.getConvertedValue(UIInput.java:1030) [jboss-jsf-api_2.1_spec-2.0.1.Final.jar:2.0.1.Final]
at javax.faces.component.UIInput.validate(UIInput.java:960) [jboss-jsf-api_2.1_spec-2.0.1.Final.jar:2.0.1.Final]
at javax.faces.component.UIInput.executeValidate(UIInput.java:1233) [jboss-jsf-api_2.1_spec-2.0.1.Final.jar:2.0.1.Final]
at javax.faces.component.UIInput.processValidators(UIInput.java:698) [jboss-jsf-api_2.1_spec-2.0.1.Final.jar:2.0.1.Final]
at javax.faces.component.UIComponentBase.processValidators(UIComponentBase.java:1214) [jboss-jsf-api_2.1_spec-2.0.1.Final.jar:2.0.1.Final]
at javax.faces.component.UIComponentBase.processValidators(UIComponentBase.java:1214) [jboss-jsf-api_2.1_spec-2.0.1.Final.jar:2.0.1.Final]
at org.primefaces.component.fieldset.Fieldset.processValidators(Fieldset.java:197) [primefaces-3.4.jar:]
at javax.faces.component.UIComponentBase.processValidators(UIComponentBase.java:1214) [jboss-jsf-api_2.1_spec-2.0.1.Final.jar:2.0.1.Final]
at javax.faces.component.UIComponentBase.processValidators(UIComponentBase.java:1214) [jboss-jsf-api_2.1_spec-2.0.1.Final.jar:2.0.1.Final]
at org.primefaces.component.layout.Layout.processValidators(Layout.java:245) [primefaces-3.4.jar:]
at javax.faces.component.UIForm.processValidators(UIForm.java:253) [jboss-jsf-api_2.1_spec-2.0.1.Final.jar:2.0.1.Final]
at javax.faces.component.UIComponentBase.processValidators(UIComponentBase.java:1214) [jboss-jsf-api_2.1_spec-2.0.1.Final.jar:2.0.1.Final]
at javax.faces.component.UIViewRoot.processValidators(UIViewRoot.java:1172) [jboss-jsf-api_2.1_spec-2.0.1.Final.jar:2.0.1.Final]
at com.sun.faces.lifecycle.ProcessValidationsPhase.execute(ProcessValidationsPhase.java:76) [jsf-impl-2.1.7-jbossorg-2.jar:]
at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101) [jsf-impl-2.1.7-jbossorg-2.jar:]
at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118) [jsf-impl-2.1.7-jbossorg-2.jar:]
at javax.faces.webapp.FacesServlet.service(FacesServlet.java:593) [jboss-jsf-api_2.1_spec-2.0.1.Final.jar:2.0.1.Final]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) [jbossweb-7.0.13.Final.jar:]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.13.Final.jar:]
at org.jboss.weld.servlet.ConversationPropagationFilter.doFilter(ConversationPropagationFilter.java:62) [weld-core-1.1.5.AS71.Final.jar:2012-02-10 15:31]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280) [jbossweb-7.0.13.Final.jar:]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.13.Final.jar:]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) [jbossweb-7.0.13.Final.jar:]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) [jbossweb-7.0.13.Final.jar:]
at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) [jbossweb-7.0.13.Final.jar:]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) [jbossweb-7.0.13.Final.jar:]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) [jbossweb-7.0.13.Final.jar:]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) [jbossweb-7.0.13.Final.jar:]
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) [jbossweb-7.0.13.Final.jar:]
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) [jbossweb-7.0.13.Final.jar:]
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) [jbossweb-7.0.13.Final.jar:]
at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_35]
