mogrify is the tool you are looking for – it comes with the great imagemagick package and it’s designed especially to be used for bulk operations.
To resize all images in current directory to 600 pixels wide (width):
I have a collection of related photos stored on my PC. The filenames were “ugly” – standard file names coming from the photo camera. I would like to rename them into a file containing the date when the photo was shoot, e.g. 2011.07.21.jpg. There is an excellent commandline tool to do just that – renrot. It’s packaged for Debian & Ubuntu so to install it, I’ve simply run:
apt-get install renrot
Then, in the directory that contains my photos I’ve run:
renrot -n"%Y.%m.%d"*
-n option allows for specifying the template used for renaming the files – it is year, month and day separated with dots in my case. If the tool creates more files, you can remove the originals:
In this blog we’ll be looking at how to use hibernate tools with just ant. This may be useful if you need to plug some hibernate tasks into your build system or have them scripted in any way. We will create a stand-alone ant build file with the minimum number of required libraries.
Libraries
I’ve extracted hibernate tools jar from eclipse hibernate plugin. You can find it in your eclipse installation directory, under plugins: plugins/org.hibernate.eclipse_X.X.X.X/lib/tools. Take hibernate-tools.jar and freemarker.jar. From plugins/org.hibernate.eclipse_X.X.X.X/lib/hibernate grab more jars:
dom4j-1.6.1.jar
log4j-1.2.15.jar
slf4j-api-1.5.8.jar
slf4j-log4j12-1.5.8.jar
You will also need org.apache.commons.logging.jar and JDBC driver. I use MySQL, so I’ve added mysql-connector-java-5.1.13.jar as well. Put all the jars into “lib” directory. Create other directories: classes, src and generated. Or don’t bother – download the whole lot as I’ve set it up.
Java entities
We will use two sample entity classes (POJOs) for that example: Password and Resource. A Resource can contain many Passwords.
publicclassPasswordimplementsSerializable{privatestaticfinallongserialVersionUID=1L;privateintid;privateStringpassword;// many-to-one association to ResourceprivateResourceresource;publicPassword(){}publicintgetId(){returnthis.id;}publicvoidsetId(intid){this.id=id;}publicStringgetPassword(){returnthis.password;}publicvoidsetPassword(Stringpassword){this.password=password;}publicResourcegetResource(){returnthis.resource;}publicvoidsetResource(Resourceresource){this.resource=resource;}}
<?xml version="1.0"?><!DOCTYPE hibernate-mapping PUBLIC
"-//Hibernate/Hibernate Mapping DTD 3.0//EN"
"http://hibernate.sourceforge.net/hibernate-mapping-3.0.dtd"><hibernate-mapping><classname="net.opensesam.entity.Password"table="password"><idname="id"type="long"column="ID"/><many-to-onename="resource_id"class="net.opensesam.entity.Resource"/><propertyname="password"><columnname="password"/></property></class></hibernate-mapping>
<?xml version="1.0"?><!DOCTYPE hibernate-mapping PUBLIC
"-//Hibernate/Hibernate Mapping DTD 3.0//EN"
"http://hibernate.sourceforge.net/hibernate-mapping-3.0.dtd"><hibernate-mapping><classname="net.opensesam.entity.Resource"table="resource"><idname="id"type="long"column="ID"/><propertyname="name"><columnname="name"/></property></class></hibernate-mapping>
Ant build file
Finally we can build build.xml! We’ll first add paths to the required jars, then define new task: hibernatetool. We can then use our new task via <hibernatetool> tag – see the code below, it’s pretty self-explanatory.
Remember me functionality is a mechanism that lets users log in automatically, even when they re-start their browser. In other words: after their session cookie is deleted by the browser (which is normally after closing the browser). This is implemented by setting up another cookie that will live in the browser’s database for a longer period (e.g. weeks). Spring out of the box offers two implementations of this feature. This tutorial presents two different approaches to cookie creation: hash-based token and persistent token. We will only look into the remember-me part of the Spring Security configuration, if you need more information look at basic setup.
Hash-based token
Implemented by org.springframework.security.web.authentication.rememberme.TokenBasedRememberMeServices, it creates a base64 encoded cookie that contains:
username – you will be logged in as this username, if cookie is confirmed to be authentic.
expiryTime – Spring will treat cookie as invalid after expiry time. The cookie is set to be valid for 2 weeks when created.
hash – md5 hash of 2 values above, user’s password, and a key.
Hash is used to confirm that cookie is valid – e.g. no-one has tried to tweak the cookie manually. A key is a very important variable here – we set it in Spring’s configuration and it should be a unique, secret for our application. As nobody knows what key we have picked, no one will be able to generate valid hash. The above also means that:
There is no extra information stored on the server side.
If cookie is intercepted by an attacker, it can be used to log in as valid user.
Username is exposed in cookie and can be read
Cookie becomes invalid when user changes password.
The configuration itself is trivial, simply add one line with <remember-me> tag to security.xml (again, refer to the previous post for basics):
This automatically adds a new checkbox to the standard login form.
Once you login, notice additional cookie created: SPRING_SECURITY_REMEMBER_ME_COOKIE.
To test functionality, delete JSESSIONID cookie and refresh. Normally that would effectively result in a logging out but with our “remember me” cookie, new JSESSIONID is created straight away.
Persistent token
A better solution to remember me cookie is to use totally random string that will be stored as a cookie and has no meaningful data in itself. The disadvantage is that it requires storing some information on the server side. When token is presented from user’s browser, it is matched against a record in table with all tokens. Our Spring configuration will change only a bit.
As you can see there are actually two new pieces there: series and token. Both are random strings and both are passed back to user and stored in the cookie. The idea here is that token will be changed on every “remember me” login, so when a user comes with valid series but invalid token – the chances are that the cookie has been stolen and used for authentication! We could not prevent stealing the cookie but we can at least:
warn user
invalidate all tokens for current user
And this is exactly what Spring Security is doing.
Sometimes it might be very convenient to be able to debug application deployed on web server. This tutorial presents how to do it, if the application is deployed using Jetty plugin for Maven or Tomcat servers.
In the tutorial below we will look at a basic integration of a Java web application with Spring Security 3. In this particular scenario we’ll be integrating Spring Security with existing web application that uses Struts & Spring. However, similar steps can be applied to any other web framework you may be using. We will also look at one of the core concepts Spring Security – AccessDecisionManager.
Basic integration
We will start with an application that has no security implemented. I’ve used this setup as my base. It already uses maven, so the first thing we do is to add a new repository:
Spring Security is implemented as a filter – it can be plugged into the application without any changes. A request will be intercepted by DelegatingFilterProxy and authentication will take place, as defined in Spring security XML file. We will add filter to the top of web.xml and trigger it for all URLs (url pattern set to /*).
We are defining here that access to any resource (intercept-url pattern=”/*”) should be allowed only for users that have authority called ROLE_USER. Normally user credentials will need to be stored in a central place – typically a database or LDAP server. Above we are hard-coding an authentication-provider with one user (guest/test) that has authority ROLE_USER.
Finally, we need to tell Spring about security.xml. Since our application already uses Spring, we have defined in web.xml contextConfigLocation parameter already – it points to our beans definition (applicationContext.xml). We will add security.xml in the same place:
When we build the application now, any URL should automatically generate following login page:
Access decision manager
Spring Security deals (or helps you deal with) two security concepts: first authentication and then authorization. Authentication filters are run first – they will make sure that a user is who he says he is. This is usually done by checking user credentials, e.g., username and password). Let’s skip authentication step for now and talk about authorization. Authorization takes care of deciding if authenticated user (meaning that authentication must precede authorization) should be allowed to access a resource.
Spring Security supports authorization with the help of AccessDecisionManager interface. AccessDecisionManagers in turn will use one or more AccessDecisionVoters. Each voter will make the decision about granting or denying the access for given user to given resource. Access decision manager will aggregate the votes and issue final decision. Voters can return: ACCESS_GRANTED,ACCESS_DENIED but also ACCESS_ABSTAIN if a voter can’t or doesn’t want to make a decision. Managers must make the final decision – user is either granted access or not. The “not granting the access” part is done by throwing an exception – usually AccessDeniedException. In the default configuration that we’ve created, Spring uses implementation of AccessDecisionManager called AffirmativeBased. It will grant the access if any voter returns ACCESS_GRANTED. The default voters are
RoleVoter – checks GrantedAuthority (remember user tag with authorities=”ROLE_USER”) against ConfigAttribute (that comes from intercept-url pattern=”/*” access=”ROLE_USER”)
AuthenticatedVoter – can check if user IS_AUTHENTICATED_FULLY, IS_AUTHENTICATED_REMEMBERED or IS_AUTHENTICATED_ANONYMOUSLY. This is not used in our setup
We can see authorization in working when we enable higher logging level – simply add to log4j properties:
When we try to access protected page as anonymous user, among other messages we can see following entries, with AccessDeniedException thrown at the end. RoleVoter returns ACCESS_DENIED and AuthenticatedVoter returns ACCESS_ABSTAIN.
DEBUG|2011-07-1012:40:30,473|AffirmativeBased.java|decide|53|Voter:org.springframework.security.access.vote.RoleVoter@1ae73783,returned:-1DEBUG|2011-07-1012:40:30,474|AffirmativeBased.java|decide|53|Voter:org.springframework.security.access.vote.AuthenticatedVoter@41ed8741,returned:0DEBUG|2011-07-1012:40:30,478|ExceptionTranslationFilter.java|handleException|153|Accessisdenied(userisanonymous); redirecting to authentication entry point
org.springframework.security.access.AccessDeniedException:Accessisdenied
Compare it with the log after we provide correct credentials. This time RoleVoter returns ACCESS_GRANTED and as soon as it happens, AffirmativeBased access decision manager stops querying voters and allows for access:
When developing web application set up of convenient development environment might be crucial for more efficient work. Let’s consider web application that uses following frameworks:
This tutorial presents initial setting of Eclipse project to support all the frameworks mentioned above and allow the convenient and fast application development of web application on Jetty server.
[ad#adsense]
Create Eclipse project
First step is to create Ecplise project, e.g., Dynamic Web Project, that would have the following structure:
Make sure that ‘src/main/java’ and ‘test/main/java’ are your source folders on building path and default output folder is ‘target/classes’.
Complete Eclipse project can be downloaded here:
Configure Maven
Maven configuration is in ‘pom.xml’ file. It consists of:
list of ‘repositories’
list of ‘pluginRepositories’
list of ‘dependencies’
‘build’ settings, i.e., ‘resources’ and ‘plugins’
list of ‘plugins’
Make sure that:
‘packaging’ is set to ‘war’
Crucial plugins
maven-war-plugin. The WAR Plugin is responsible for collecting all artifact dependencies, classes and resources of the web application and packaging them into a web application archive.
<plugin><groupId>org.apache.maven.plugins</groupId><artifactId>maven-war-plugin</artifactId><configuration><archive><manifest><addClasspath>true</addClasspath><classpathPrefix>lib/</classpathPrefix></manifest></archive><webResources><resource><!-- this is relative to the pom.xml directory --><directory>src/main/webapp</directory><targetPath>/</targetPath><!-- enable filtering --><filtering>false</filtering></resource></webResources></configuration></plugin>
maven-jetty-plugin. In order to run Jetty on a webapp project which is structured according to the usual Maven defaults.
Configuration of ‘log4j.properties‘ is presented below:
### direct log messages to stdout ###
log4j.rootLogger=ERROR, file
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.Target=System.out
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%6p | %d | %F | %M | %L | %m%n
# AdminFileAppender - used to log messages in the admin.log file.
log4j.appender.file=org.apache.log4j.FileAppender
log4j.appender.file.File=target/SpringSecurity.log
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%6p | %d | %F | %M | %L | %m%n
log4j.logger.net.zabuchy=DEBUG, console
Configure Spring beans
Configuration of Spring beans should be placed in ‘applicationContext.xml’ file. In this project Spring beans configuration files were included in ‘applicationContext.xml’ file. ‘beans.xml’ defines ‘outcomeAction’ bean, which represents Struts2 action and other beans required by this action, i.e., ‘outcomeDeterminer’. ‘databaseBeans.xml’ includes configuration of beans required by Hibernate. ‘dataSource’ bean includes configuration of database to be used in this project. It should be changed accordingly. ‘sessionFactory’ bean includes configuration of Hibernate like dataSource (pointing to ‘dataSource’ bean) and packagesToScan for @Entity beans. Content of the files is presented below.
<?xml version="1.0" encoding="UTF-8"?><beansxmlns="http://www.springframework.org/schema/beans"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd"><!-- Data source configuration --><beanclass="org.springframework.jdbc.datasource.DriverManagerDataSource"id="dataSource"><propertyname="driverClassName"value="com.mysql.jdbc.Driver"/><propertyname="url"value="jdbc:mysql://127.0.0.1:3306/db"/><propertyname="username"value="test"/><propertyname="password"value="test"/></bean><!-- Hibernate session factory --><beanclass="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean"id="sessionFactory"><propertyname="dataSource"><refbean="dataSource"/></property><propertyname="hibernateProperties"><props><propkey="hibernate.dialect">org.hibernate.dialect.MySQLDialect</prop><!-- In prod set to "validate", in test set to "create-drop" --><propkey="hibernate.hbm2ddl.auto">update</prop><!-- In prod set to "false" --><propkey="hibernate.show_sql">false</prop><!-- In prod set to "false" --><propkey="hibernate.format_sql">true</prop><!-- In prod set to "false" --><propkey="hibernate.use_sql_comments">true</prop><!-- In prod set to "false", in test set to "true" --><propkey="hibernate.cache.use_second_level_cache">false</prop></props></property><propertyname="packagesToScan"value="net.zabuchy"/></bean></beans>
Configure Struts2 actions
Struts2 actions are defined in ‘struts.xml‘ file, which is presented below. ‘getOutcome’ action uses ‘outcomeAction’ bean configuration and invokes ‘getOutcome’ method from this bean. If ‘getOutcome’ method returns ‘success’,'success.jsp’ will be displayed. If ‘getOutcome’ method returns ‘error’,'error.jsp’ will be displayed.
Sample Java classes code, which was used in this project is presented below. Two classes were created and defined as Spring beans in ‘beans.xml’ file: ‘OutcomeAction’, which is Struts2 action and ‘OutcomeDeterminer’, which is class used by this action.
‘OutcomeAction.java‘ file in package net.zabuchy.actions
‘OutcomeDeterminer.java‘ file in package net.zabuchy.business
packagenet.zabuchy.business;importjava.util.Random;importorg.apache.log4j.Logger;publicclassOutcomeDeterminer{privatestaticfinalLoggerlogger=Logger.getLogger(OutcomeDeterminer.class.getName());privateRandomgenerator;publicOutcomeDeterminer(){generator=newRandom();}publicStringgetOutcome(){intrandomNumber=generator.nextInt(5);if(logger.isDebugEnabled()){logger.debug("Random number is: "+randomNumber);}if(randomNumber==0){return"error";}return"success";}}
Add libraries to Eclipse
The easiest way to add required libraries to Eclipse is allow Maven to download all the required libraries (defined as dependencies in pom.xml) and add them to ‘Referenced Libraries’. In order to do so in root directory of the application run:
mvn package
This command should build war package to be deployed on web server including:
downloading required libraries and coping them to ‘target/{projectName}/WEB-INF/lib’ (configured in ‘manifest’ section of maven-war-plugin)
compiling classes and coping them to ‘target/{projectName}/WEB-INF/classes’
coping files from ‘src/main/resource’ to ‘target/{projectName}/WEB-INF/classes’
coping files from ‘src/main/webapp’ to ‘target/{projectName}’ (configured in ‘webResources’ section of maven-war-plugin)
In Eclipse add libraries from ‘target/{projectName}/WEB-INF/lib’ to build path. It might be required to move them to the top ‘Order and Export’.
Run application
We are going to use jetty server to run the application. ‘jetty:run’ command allows the automatic reloads, so during development the changes done in Eclipse are automatically deployed to the server. In order to run the application type:
mvn jetty:run
maven-jetty-plugin automatically scans ‘src’ (configured in ‘scanTargets’ section of maven-jetty-plugin) every 5 seconds (configured in ‘scanIntervalSeconds’ section of maven-jetty-plugin) for changes.
To deploy the project on different web server, e.g., Tomcat, just use ‘mvn package’ command and copy war file to appropriate directory on your Tomcat server.
In your web browser access the following link: ‘http://localhost:8080/SpringSecurity/getOutcome.action’. You should be able to see either success or error page.
Sometimes simply checking the size of the whole directory is not enough, and you need more information, e.g. the whole directory has 3GB – but how much of it are those .avi files inside? The most flexible way for matching the files (or directories) is by using find command, so let’s build the solution based on this. For the actual counting of the file sizes, we can use du.
Consider directory as follows. Numbers in the brackets are the file sizes.
Size of the whole directory (-b option gives the size in bytes):
% du-sb filesizes
121227 filesizes
Total size of selected files
Let’s calculate the size of all the yml files. They are stored in different sub-directories and one of them contains a whitespace in the name (blah space.yml). find part of the command is straight-forward:
% find filesizes -type f -name'*yml'
filesizes/dir1/blah space.yml
filesizes/config.yml
Find all files (-type f) with the name ending with yml (-name ‘*yml’) in filesizes directory (that’s first argument – if it’s omitted then find will work from the current directory).
To join it with du command we will change the output of find to separate files it finds with NULL character (0) instead of default newline. We will also pass the list of files to du command, and instruct it that the items (files) are coming in the NULL-separated list from standard input:
% find filesizes -type f -name'*yml'-print0 | du-cb--files0-from=-
5 filesizes/dir1/blah space.yml
290 filesizes/config.yml
295 total
-c switch for du generates the last line, with the total count.
Total size of selected directories
Let’s try something a (little) bit more tricky – calculate the total size of all sub-directories, matched by a pattern. In our example, I would like to know the total size of all backup directories. Notice, that dir3 has backup sub-directory, which in turn has backup sub-directory. We don’t want to count those as two separate entries and sum them up by accident – that would bump our total size (you could possibly get the size of all backup directories bigger than the size of the whole top level directory!).
% find filesizes -type d -name backup -prune-print0 | du-csb--files0-from=-
24679 filesizes/dir1/backup
49358 filesizes/dir3/backup
24679 filesizes/dir2/backup
98716 total
This time we’re find-ing only directories (-type d) and once we find one, we stop and don’t go into it’s sub-directories (-prune). On the du side we’ve added -s switch that produces the summarized total for each item (directory) it gets.
One of the easiest methods of optimizing the execution of PHP script is by using a static variable. Imagine a function that does some complex calculations, possibly executes several database queries. If the return value from the function does not change too often and the function may be called several times during single request, it may be beneficial to cache the return value with static variable,
You could implement it within your complex function or with a wrapper function if you’re trying to cache the function you didn’t write.
functionmymodule_complex_calculation(){static$cache;if(isset($cache)){return$cache;}// Otherwise we need to perform our calculations// ...some complex calculation that returns $cache=complex_calculation();return$cache;}
This is a very simple pattern and cached data will “survive” only single web request but it may still be very effective, especially if function is called many times during the same request. Drupal implements a drupal_static() function that is a serves as centralized place for storing static variables. The advantage over using your own static variable is that data stored by drupal_static() can be reset. It means that, if other piece of code decides that some cached variable should be cleaned in mid-request, it can do so. Function definition is:
You will need a unique name to address the data you want to cache. This will really become a key in the static array. The customary way in Drupal is to use the name of your function as $name, you can use PHP’s magic constant __FUNCTION__ that resolves to current function name. Now we can update our function as follows:
functionmymodule_complex_calculation(){$cache=&drupal_static(__FUNCTION__);if(isset($cache)){return$cache;}// Otherwise we need to perform our calculations// ...some complex calculation that returns simple value$cache=complex_calculation();return$cache;}
You may notice that we don’t use drupal_static() to actually set the data – and we don’t need to! As soon as we assign anything to $cache, it will end up in our centralized static store. drupal_static() has one disadvantage – it adds a new function call (plus execution of some logic inside drupal_static()) every time you use cache which increases the execution time. Compare it with the listing 1 – much simpler. This will add some overhead to each call. At this level we are talking about micro optimizing. It will make noticeable difference only if you function is called hundreds (or thousands) of times per request. If that’s the case, you can use advanced usage pattern for drupal_static. It merges functionality of our 2 functions:
functionmy_funct(){static$drupal_static_fast;if(!isset($drupal_static_fast)){//17 is the default value in case the cache was empty$drupal_static_fast['cache']=&drupal_static(__FUNCTION__,17);}$cache=&$drupal_static_fast['cache'];//do whatever you need with $cache ...
Pay attention to the fact that $drupal_static_fast is an array and the actual cache is within that array ($drupal_static_fast['cache']). You have to use an array, otherwise your reference to the static variable will be lost and each time my_funct() is called, it will need to retrieve a value using drupal_static() – therefore negating our optimization (see manual for the details).
I’ve added a benchmark for it to micro-optimization site. Assuming I’ve got the test right, you can see the 17x difference between accessing static variable vs calling drupal_static() function.
Apache log4j is one of the most popular frameworks used for logging events within Java code. Apache CXF on the other hand is one of the most popular framework to support communication using web services.
To monitor and debug Java application there might be a need to log inbound and outbound web service messages.
If we use two frameworks mentioned above org.apache.cxf.interceptor.LoggingInInterceptor and org.apache.cxf.interceptor.LoggingOutInterceptor might be configured without any intergeneration in the code to log web service messages.
In the following configuration web service messages will be logged to console on INFO level.