Query optimization in Orientdb

If you have used Orientdb with quite large dataset you may have experienced a huge performance decrease in query execution when the number of records increases. You may also seen tips from orientdb saying the number of record reads is above 50000 or some number and use some index to reduce the number of record reads.

The reason for this type of behavior is it has some limitations when choosing for effective indexes. And when the number of record reads it has to perform the performance reduces drastically. Let me explain the scenario with an example.

Consider a class A, with more than 100000 records and if it has N attributes as attr1, attr2, .... attrN. Also assume that attr1 has cardinality of 1, attr2 has cardinality of 25000, attr3 has cardinality of 50000 likewise. We also have indexes for each and every attributes seperately. (For the time being lets not consider about composite indexes)

So if we execute a query like this;

select from A where attr1 = 'aaa' and attr2 = 'bbb' and attr3 = 'ccc' ...

If you use explain command from the orientdb studio you may see orientdb is applying only the index on attr1. This is because attr1 is the first in the where condition of the select query. Since it is an attribute with very low cardinality the number of document reads will be very high. Therefore the index is almost useless in this kind of situations.

So if you want to optimize the performance of this query try to get the attribute with very high cardinality first where the number of document reads will be reduced.

Furthermore if you want to enforce the use of composite indexes use parenthesis like this.  

select from A where (attr1 = 'aaa' and attr2 = 'bbb') and attr3 = 'ccc' ...

Hope this will be helpful to some one who is struggling with orientdb query optimization.

Note: Above explanation is done for the orientdb version 2.1.1.

How to integrate tomcat to a LDAP authentication

After struggling 3 days finally I was able to integrate a tomcat server to a LDAP server.

With LDAP you will be able to authenticate users to a web application with existing LDAP server.


The basic steps goes as follows.

For this example I have used an online LDAP test server hosted at (http://www.forumsys.com/tutorials/integration-how-to/ldap/online-ldap-test-server/).

First you edit your server.xml file at your <catelina_home>/conf/server.xml as follows.
Comment out the UserDatabaseRealm class and add the JNDIRealm part there.

<!-- Use the LockOutRealm to prevent attempts to guess user passwords
           via a brute-force attack -->
      <Realm className="org.apache.catalina.realm.LockOutRealm">
        <!-- This Realm uses the UserDatabase configured in the global JNDI
             resources under the key "UserDatabase".  Any edits
             that are performed against this UserDatabase are immediately
             available for use by the Realm.  -->

        <!--Realm className="org.apache.catalina.realm.UserDatabaseRealm"
               resourceName="UserDatabase"/-->
               <Realm  className="org.apache.catalina.realm.JNDIRealm" debug="99"
             connectionURL="ldap://ldap.forumsys.com:389"
             connectionName="cn=read-only-admin,dc=example,dc=com"
             connectionPassword="password"
             userPattern="uid={0},dc=example,dc=com"
    />     
      </Realm>

Note: you can use JNDIRealm class without LockOutRealm also. Lockout realm is used to lock the users if they enter the password several times incorrectly.

Note: If you set 3268 as your port in ldap you can access global catalog where you can access outside of the user pattern path also.

Then goto your maven web project and goto the web.xml file at <catelina_home>/webapps/<proj-name>/WEB_INF/web.xml and add the followings

<security-constraint>
      <display-name>Example Security Constraint - part 1</display-name>
      <web-resource-collection>
         <web-resource-name>Protected Area - Allow methods</web-resource-name>
         <!-- Define the context-relative URL(s) to be protected -->
         <url-pattern>/<path from cat_base>/*</url-pattern>
         <!-- If you list http methods, only those methods are protected so -->
         <!-- the constraint below ensures all other methods are denied     -->
         <http-method>DELETE</http-method>
         <http-method>GET</http-method>
         <http-method>POST</http-method>
         <http-method>PUT</http-method>
      </web-resource-collection>
      <auth-constraint>
         <!-- Anyone with one of the listed roles may access this area -->
         <role-name>*</role-name>        
      </auth-constraint>
    </security-constraint>
  

    <!-- Default login configuration uses form-based authentication -->
    <login-config>
      <auth-method>BASIC</auth-method>
    </login-config>

    <!-- Security roles referenced by this web application -->

    <security-role>
      <role-name>*</role-name>
    </security-role>

Here role name * means an user with any role can login to the system.

Here I have used the basic authentication method which means you do not need to implement a login page. If you need to implement a login page use form based authentication method.

That's it. now you have successfully implemented your authentication system. Further more if you want to debug the tomcat authentication steps add the following lines to <catelina_home>/conf/logging.properties file

############################################################
# Facility specific properties.
# Provides extra control for each logger.
############################################################
# This would turn on trace-level for everything
# the possible levels are: SEVERE, WARNING, INFO, CONFIG, FINE, FINER, FINEST or ALL
#org.apache.catalina.level = ALL
#org.apache.catalina.handlers = 2localhost.org.apache.juli.FileHandler
org.apache.catalina.realm.level = ALL
org.apache.catalina.realm.useParentHandlers = true
org.apache.catalina.authenticator.level = ALL
org.apache.catalina.authenticator.useParentHandlers = true


Now go to your web application and it will ask for a password. Use either of following names and try password as password.

• riemann
• gauss
• euler
• euclid
• einstein
• newton
• galieleo
• tesla

Special thank goes to Forum system people for hosting online LDAP server for testing purposes

Ubuntu : Could Not Apply The Stored Configuration For Monitors

In the last couple of days I got a message in ubuntu when I log into the machine saying,

"Could Not Apply The Stored Configuration For Monitors and blah blah blah...." 

And all my fonts and interfaces look ugly.

So I looked into the matter and found this thread.

http://ubuntuforums.org/showthread.php?t=1852813

The solution was quite simple.

1. goto home folder 
2. (press ctrl + h) and goto the folder .config 
3. delete (or if you are not sure just rename) monitor.xml file
4. restart (or just log out and log in will solve the matter I guess)
5. enjoy.. :)

Installing Wijesekara layout in Ubuntu 12.04 LTS

Since I'm a huge fan of Wijesekara layout sinhala keyboard, I struggled little bit to get it running from the guidelines given by Siyabas (http://www.siyabas.lk/sinhala_how_to_install.html#lin) site.

Even after setting the universe settings from the ubuntu software sources, it always gave me a message saying ibus-m17n package is unable to locate.

Then I found this thread (http://askubuntu.com/questions/105544/install-sinhala-unicode-pack) from the ask ubuntu site and there was a answer given by Maduka Jayalath saying ibus is the solution.

So I followed the link (https://help.ubuntu.com/community/ibus) and installed ibus-m17n separately.

sudo apt-get install ibus-m17-n

Finally I executed the instructions given by Siyabas site again and it successfully enabled Sinhala unicode Wijesekara layout in my Ubuntu 12.04 LTS

Incrementer using Combining tree

This video shows an example of parallel counter operation implemented using a combining tree.
This has O(lgn) complexity when run in parallel while the normal counter with locks will have a complexity of O(n)

Bakery Algorithm

https://www.youtube.com/watch?v=sfVKdT0TgLQ

Project is now on GitHub

Now you can view the project on github.
https://github.com/lasitha1991/VLC-Security-IP-Solution