Friday, October 2, 2009

SQL Server 2008 Memory Leak / windows server 2008 high memory utilization

Having working with Cubes on SQL Server 2000, I had grown used to the fact that SQL will use and release memory depending on its need and I would be able to track the about of memory each program is utlizing through the task manager tab.

However its important to note the following. With windows server 2008 and SQL Server 2008 ( probably SQL Server 2005) the above mentioned assumptions are completely wrong. SQL Server 2008 will utilize as much memory that is available on the box, in my case 64 GB if any job works with high volumes of data but will NEVER EVER release this RAM back to the system once the job completes.

In short, the memory get reserved for SQL Server 2008 till the service / Server gets restarted. In case SQL needs to run some other job that requires data to be loaded, it internally will flush out the older data and load what it needs and yes even then nothing gets released back to the server.

Also to be noted is that in the task manager, none of this memory is ever shown against any of the processes that are running. Even if you use process explorer (sysinternals) this memory does not show up against SQL Server 2008 there as well.



Sadly this leads people ( including me) to come to the conclusion that there is a memory leak in SQL Server 2008. To convince yourself, just limit the amount of memory SQL Server can use and then check out the memory utilization in task manager. It will be that amount plus whatever is required to run your server.

There is article that describes this much more technical detail here ...

http://sqlblog.com/blogs/jonathan_kehayias/archive/2009/08/24/troubleshooting-the-sql-server-memory-leak-or-understanding-sql-server-memory-usage.aspx


Its a good idea to prevent SQL from utilizing the entire memory else the server can become unstable.

P.S. There is no memory leak :)

Saturday, August 22, 2009

Excel 2007 OLAP Writeback SSAS 2008

We wanted to build an application based on the Writeback capability of Microsoft. The dream was to have a cube with multiple dimension allowing our users to analyze data by slicing / dicing this cube and based on their analysis make changes to the numbers at various levels.

Once they were happy with the data post changes we were to flow this data to a downstream system. It was to be a highly interactive system for a reasonably competent user to plan target setting by carrying out what if analysis on the existing data.

The front end for the user was Excel 2007 as the navigation / user interface was phenomenon and all our users in our company were fairly competent in Excel.

The path we started down was to have SQL Server Analysis Services 2005 as the backend sitting on a Hexacore 4 processor Windows Server 2008 box. We then enabled the cube for Writeback. The way this works according to me at a high level is that whatever numbers you choose to writeback, it does not make the change directly to your cube, what it does however is to create a writeback partitition that stores the net off data so that you could see the required number on your front end.

Let me explain that a little more with an example, lets say for one depot for one product the value initially was 100. As part of the writeback lets say you want it to be 70. What actually happens is that an entry takes place in the writeback partition for that depot and that product as -30. Based on a join of these two the number dispayed is 70.

While this works perfectly for small sets of data, it fails miserably for larger sets. The solution becomes impractical and very sluggish if the writeback is at an aggregate level and affecting large number of leafs / touples. Post engaging with Microsoft Consulting we confirmed that this was not going to work out for the application we had in mind.

OLAP Writeback sucks, period !!! and even Microsoft didn't have a response :) Now just because the writeback using the writeback partitition is hopeless it does not mean that its the only way to writeback user inputs into the cube.

The path we finally deployed our solution was a combination of the following.
VBA ( Macro coding) / Web services / SQL Server 2008 Integration services and SQL Server 2008 Analysis Services.

The excel that was shared with the users had a custom built macro code that essentially looped through the page / row / column fields to gather the user selections. Care had to taken to be able to loop through both the dimension member as well as the attributes. ( This is feature of SQL Server 2005 + , do read up if you are unaware) Also note it gets more complicated in Pivot 12 as there is extended capability in terms of Value and Label filters, both of which needs to be captured as well incase someone has applied them.

Once all the filters are looped and the details captured, the macro uses this data to frame a create sub cube query that gets passed to the server using a web service. This sub cube that gets created on the fly contains the entire data from the main cube that needs to undergo the writeback / change.

The next step is to strip this newly created sub cube to retrieve the underlying rowset of data that makes up the cube. Once we have that into a table, we have the option of applying pretty much any rule on this data, disaggregation / static values / percentage of existing value / percentage of another value.

Post applying the rule, the data gets updated into the base cube data and an incremental process is carried out and voila you have implemented your writeback.

Just to share some of the numbers, we were able to carry out a writeback affecting 3 Lakh records ( Thats three hundred thousand records :) in India) in a little over 2 minutes with the majority of the time in the incremental processing. This ofcourse will depend on the size and complexity of your own cube.

If your curious about the rest of the design, well since we had the changes in the table we could build an SSIS package to incrementally flow this information down stream into a DB2 environment using OLE/DB2 providers which ofcourse has its own problems in life :)

Sunday, June 21, 2009

Motorola RAZR V3 - Charge via USB on PC

There are other articles that explain this but finding the files for this can sometimes be a problem. Basically the usb does not work directly, software drivers need to be installed on the machine that you want to use to charge.

These drivers can be downloaded here ...

http://ofg.osnn.net/razr/


The detailed article for the same can be found here

http://www.somelifeblog.com/2007/02/motorla-razr-v3-charge-via-usb-on-pc.html

Tuesday, May 12, 2009

IBM OLE DB provider for DB2 sql server 2008 / DB2 Administration Client Windows Server 2008

While setting up our production environment we ended up wasting a day as we couldn't create a linked server using IBM OLE DB provider for DB2 as it wasn't showing up in the options. We apparently were trying to install the 32 bit client of the server while we needed the x64 version. After figuring that out we began a hunt to download the required file which in reality was quite painful as it wasn't clearly mentioned anywhere.


In order to save others the trouble, the driver comes along with the installation of the "DB2 Administration Client " as part of the 8.2 versions and as "IBM Data Server Client " in the 9.5 versions. The links in the document are valid for Windows Server 2008 utilizing x64 architecture. If your on 32 bit , you can navigate to the right download from the given pages itself.

The page for the current version of DB2 v9.5 is as follows, make sure you download the "IBM Data Server Client" file. It is part of the fix pack 3b which is latest at the time of this article

http://www-01.ibm.com/support/docview.wss?rs=71&uid=swg21288113

The direct file

ftp://ftp.software.ibm.com/ps/products/db2/fixes2/english-us/db2winX64v95/fixpack/FP3b_WR21447/v9.5fp3b_ntx64_client.exe


In case you are using / want the DB2 v8.2 client, make sure you download the "DB2 Administration Client" file. It is part of the fix pack 17a which is latest at the time of this article

http://www-01.ibm.com/support/docview.wss?rs=71&uid=swg21256105


The direct file

ftp://ftp.software.ibm.com/ps/products/db2/fixes2/english-us/db2winAMD64v8/fixpak/FP17a_WR21442/FP17a_WR21442_ADMCL.exe


Please note that there are different versions for different architectures x32 / x64 / IA64. Download the right file for yourself.

Additionally if you are looking for the Microsoft OLEDB Provider for DB2, it is available as part of the Microsoft SQL Server 2008 Feature Pack, October 2008 located here

http://www.microsoft.com/downloads/details.aspx?FamilyId=228DE03F-3B5A-428A-923F-58A033D316E1&displaylang=en


Do let me know if this has been helpful. Cheers !

Thursday, April 9, 2009

Google Chrome not working / Google Chrome stopped working

I had this really weird issue where google chrome stopped working on my laptop, did some search and I finally found this obscure post on google's own site asking the user to try something to make it work. Interestingly it did, the version for which the solution was written for was different but essentially the method is what is important.

The issue was that on clicking the chrome icon nothing happens, the browser does not load up, and the task manager does not show the process to be running. Not sure what triggered this though.


The solution is to go to the following path in your command prompt

C:\Documents and Settings\(user name)\Local Settings\Application Data\Google\Chrome\Application\(version number)\Installer

Run the following statement

setup.exe --rename-chrome-exe


quit the command prompt with an exit. Chrome started working for me after that, hope it works for you as well. The link of the google page from where I got this from is below

http://www.google.com/support/forum/p/Chrome/thread?tid=2680b41d121dc73a&hl=en


Do let me know if this has helped you, if your feeling generous you could donate a dollar via paypal :)

Saturday, March 28, 2009

Airtel blocking Websites partially

I had this unique problem on my parents laptop, certain sites such as facebook, yahoo groups, hotmail would be able to log in but thats about it, nothing else would work after that with the page just timing out when you click on anything.

Mail on the POP3 would download but SMTP mail send would fail occasionally by timing out.

Wondering why my ISP was preventing this from happening. The Airtel Beetel Modem is connected to a Netgear Router for wireless access.

It turns out that the issue was a combination of my ISP along with the wireless Router. There exists a setting on the wireless routers called MTU size in bytes. This typically is found on the WAN tab of your router and is set to 1500 bytes. The type of connection between the modem and the wireless router is a PPPoE type of connection. For this sort of connection the ideal value for this parameter is 1492. The technical details of this can be found on the following link.

http://www.dslreports.com/faq/695


By changing this parameters all the problems just disappeared ! Further read up has shown that this could cause problems to other services as well such as telnet / ftp / https / VPN like securepoint etc.

In order to change this setting, log into your wireless router and navigate to the WAN setup link. Make the change and click apply.


Do let me know if this has been helpful by dropping me a comment.

Monday, January 12, 2009

Blog Language Translation

Reading up that the majority of users the world over search for information in languages other than english it obviously makes sense to provide the option to translate my blog into different languages.

Unfortunately it wasn't as easy as I thought it would be as the widgets offered by Blogspot didn't seem to have it directly available.

I finally managed to find the translation engine provided by google itself as a widget. Go to the translate.google.com site

Select the tools tab and you will get the code for added to your widget in your site.

The code is as follows

<$source src="http://www.gmodules.com/ig/ifr?url=http://www.google.com/ig/modules/translatemypage.xml&up_source_language=en&w=160&h=60&title=&border=&output=js"><$/script>

Make sure you remove the $ between < and script in the start of the code and in the end between < and /script. I needed to add that to make this blog treat it as text and not html code :)


Additionally there was this good resource I found with additional information about language translation I found over here

http://labnol.blogspot.com/2005/11/add-language-translation-to-website.html

Hope you found this useful. Do drop me a line would love to hear from you.