Friday, July 18, 2014

MoveWindows: Restoring all windows to primary display

This is just a quick hack. There is some kind of bug in AMD drivers or maybe in Win7. When my LG 34UM95-P goes to standby, some of my windows go off-screen on the right side of my display.

I can recover some of my windows back with Move-command but in the long run it is easier just to write program that does the same. First I tried to do this with Powershell but with it seems to be easier to do with Python because it's easy to get right modules.

The script ( and program)  restores all windows that are outside the primary display back to display. I haven't tested it yet with multiple displays and I'm quite sure it doesn't work. I only need this right now for one display.

If script is run without parameters, it only executes once. If it run with a parameter, the script loops indefinitely and the parameter is the delay between loops in seconds.

The source code for my scripts is in GitHub


Tuesday, July 15, 2014

Semi-dynamic Templates for Putty

One of the missing features of Putty is templates.
I've been doing some work outside and in sunlight the usual terminal colors, black background and white letters just don't work. So I had to change the settings for a couple of my sessions.

I'd been wondering a while about templates in Putty so I wrote a simple script that allows to keep your own common settings in Templates registry key. Putty has it's setting in HKCU:\Software\SimonTatham\PuTTY and all the session settings are in Sessions-key.

This script searches for all Sessions and if it finds property TemplateName it applies templates found in that property from key HKCU:\Software\SimonTatham\PuTTY\Templates. TemplateName can have more than one template name, the template names are comma-separated and applied in that order listed.

Templates can have parent templates. It as also possible to create subkeys in Templates key if you want to create your own set of templates and import that key to another computers without messing templates in there. Then the tempalate name would be like Marko/Summer.

If you import templates from others keep in mind there is no security. Everything that is in that template overwrites those properties in your session. So you should always know what settings these templates change.

When you want to apply the templates to sessions, you simply run script ApplyPuttyTemplates.ps1. The script copies all the properties found in templates over the sessions properties. (=Take a backup before you start playing with this )

There are example templates in file PuttyTemplates.reg.
Root-template has the settings you want to your every session.
Winter and Summer - templates change the screen colors and have Root template as a parent template.
Then there is a separate Season - template that has Winter or Summer as a parent template so you can quickly change Winter/Summer all the sessions that use Season template.

You can also create templates like Production, Testing that have different fonts or colors and apply those those templates after the Season template is applied (for instance  TemplateName=Season,ServerType/Production ).

It's not perfect, it's not polished. It's working and I can quickly apply new templates to my sessions.

The source code for my scripts is in GitHub

Sunday, June 15, 2014

LG 34UM95-P review :Would be great without the Super Energy Saving bug: Part 2

Calibrating with DispcalGui

I wrote a separate post about calibrating with dynamic contrast because it isn't specific to this monitor:
Calibrating with ArgyllCMS&DispCalGUI with dynamic contrast or projector iris. 

I calibrated the monitor yesterday but only with 88 patches just to check the calibrating works. Today I'm going to do the real >1000 patch calibration.

Auto Standby bug(?)

After I'd been using the monitor for 4 hours, it popped up message saying monitor hasn't been used for 4 hours and it's going to be shut down in 5 minutes. What? I've been using the monitor the whole time! And after 4 minutes another message came stating press any key or monitor is shut down in a minute. And it did shut down the monitor.

There is a setting in Menu/Settings/Auto Standby that was set to 4 hours. But in the manual it clearly says: Feature that automatically turns off the monitor when there is no movement on the screen for a period of time. Clearly it was not working at all. I don't know how it decides is there movement or not.

USB3 and Serial port used for communicating


There is an USB to Serial converter TUSB3410 inside the monitor. I have it now disabled because the ASMedia USB3 drivers were messing with my motherboards USB3 (=ASMedia). But True Color Finder seems to calibrate through this port. I would have though it was using DDC/CI but now because TUSB3410 is not working it True Color Finder says cannot connect. It should be easy to crack, there can't be that many commands. Just put a com port sniffer in between.

Cleartype Sub-pixel rendering 


I tested first with Lagoms sub-pixel rendering test and then with Windows own that sub-pixel ordering is RGB. It's better to test first with Lagom because Windows only supports RGB and BGR sub-pixel orders. 

Conclusions (for today). 

I was wondering haven't any other reviewers noticed there isn't any dynamic contrast at all, or Super Energy Saving=Off actually shuts down the backlighting completely or the monitor turns off in 4 hours ( it was the default setting) no matter what you do? Or did I just get a faulty monitor?

LG 34UM95-P review :Would be great without the Super Energy Saving bug : Part 1

Calibrating with ArgyllCMS&DispCalGUI with dynamic contrast or projector iris.

When I was trying to calibrate LG's 34UM95-P, only way to calibrate was to use it's energy saving setting (=kind of slow dynamic contrast) on. The problem was it was slow. When I was measuring with HCFR I found out it needs at least 400ms delay before the luminance settles. In HCFR you can set this delay ( Latency Time) but not in ArgyllCMS&DispcalGui.

There is a commandline option in ArgyllCMS that allows you to run your own command every time the color is changed. It's main purpose is you can have your own way of showing the colors on screen. You can easily insert there also some delay.

Create file SensorDelay.bat and put it in your root ( the easiest way, it doesn't work well with spaces in path names. ).  In this batch file put complete path name and file name of sleep command you are going to use with parameters. I don't remember if Windows has any sleep command but I have several. Cyrwin has sleep with accuracy on seconds but Windows Resource Toolkit sleep you can set the delay to milliseconds so I used that one. I copied also the sleep command to root.

So my SensorDelay.bat is:
d:\sleep -m 500

The command parameter in ArgyllCMS is
-C "D:\SensorDelay.bat"

In DispCalGui you can set it in Options/Set additional commandline arguments
You only need to add the parameter to commands dispcal and dispread.

There you go! Now there is an additional 500 ms delay before each measurement.

Saturday, June 14, 2014

LG 34UM95-P review :Would be great without the Super Energy Saving bug

I got my display last tuesday and haven't had enough time to calibrate it properly. I think I need to write down the first things I found about this monitor and add more later.

The Panel


It's wide. If you look it sideways, it resembles the Star Destroyer at the beginning of Star Wars episode IV. It never ends.

But after a couple of days you get used to it and want a wider one :)
A curved panel could work for a monitor this wide. 


The Contrast

The contrast ratio is typical for IPS panel, about 1000:1 for both static contrast and ANSI contrast.


The Spectrum


I use Colormunki Photo spectrophotometer to get spectral curves and the XYZ correction matrix for i1 Lite colorimeter. For actual measuring and calibration I then use i1 Lite because it works better in low light than spectrophotometer. As a software I use ArgyllCMS, DispcalGui and HCFR.

The Color Gamut 


Green and red are a bit oversaturated and blue is the most accurate.

The RGB levels and color temperature

The default color temperature is accurate but quite high 7000K. With setting Screen/Color/Color Temp/Warm it was still 6650K so I changed the Setting to Custom and new RGB values:
Red: 50
Green: 45
Blue: 46

Now deltaE in HCFR keeps below 3.

And color temperature is quite accurate.

The gamma ( and the Super Energy Saver )

One cannot write about gamma and gray scale measuring without the Super Energy Saving feature.
What is Super Energy Saving feature? You tell me. Or maybe LG should tell us what it is and how it's supposed to work and how it actually works.

LG Manual says:
Super Energy Saving:
High = Saves energy using high-efficient Super Energy Saving feature
Low = Saves energy using low-efficient Super Energy Saving feature
Off = Disables the Super Energy Saving feature

The gamma measurement when the setting is Off and Gamma is Gamma0 which seems to be 2.4, the way I like it.

There are no readings below 20%. Why? Because when Super Energy Saving is Off and you have dark uniform screen for more than 5 seconds, LG completely dims the backlighting. Even when Super Energy Saving is OFF! This is not a feature, this is a bug. A hope they are going to fix this soon. This bug is not in Low and High settings. They do not completely dim the monitor.

If Super Energy Saving is Low or High, there's is some kind of dynamic contrast thing going on but it's working very slow and not recommended for movies. But you can't use the setting Off either because it might dim the screen if you are watching Dark Knight. Wow. What are they thinking?

Because I cannot measure the black when SES if Off and Low and High settings use this slow dynamic contrast.only way to get measurements is setting Latency time for Dynamic Iris in HCFR to 400ms. That seems to be the time it takes for this "energy saver" to adjust to new screen brightness. For a movies or games that 400ms is a awful lot of frames and time.

This is a gamma when Latency Time is 300ms.

This is gamma when Latency Time is 400ms or more or when SES is Off.

This is not a perfect calibration because of the flaws in SES but it's the best I can get.
Easy Control
- Brightness=70
- Contrast=74
- SES=Low
- Gamma=Gamma0
- Red=51
- Green=48
- Blue=54
With these values Red color is just about to be burned and the static contrast is 1034:1 and ANSI contrast 1011:1.

The dynamic contrast


LG claims in it's web site that this monitor has 5M:1 dynamic contrast ratio. How? Do they mean when the monitor shuts the backlighting completely that it counts as dynamic contrast?? Why do they use these ridiculous claims in their marketing?

LG 34UM95-P review :Would be great without the Super Energy Saving bug: Part 2

Thursday, May 29, 2014

Using FMI open historical weather data to find correlation between water and air temperature difference and wave height

Every spring when I put my snowboard to closet and start water season in kitesurfing I have this funny feeling that waves should be higher compared to wind speed.

Of course I know about inversion that happens when water and air have to much temperature difference. The cold water keeps the surface air also cold and when two layers of air have too much temperature difference they don't move anymore in the same speed but the upper warmer layer starts of flow on top of the colder layer.

In the spring time this means sometimes that even when there is wind in wind meters there is not enough wind in surface layer to kitesurf.

Even when I know this, it still gives this funny feeling that the waves should be higher in this wind. Because when inversion occurs, the wind doesn't get to touch the surface of water and create waves.

Fetching the data

Finnish Meteorological Institute FMI is opening their data and they already have weather station history data opened. For some stations it's possible to download history data from year 2004 in 10 minute resolution.For most of the stations they have data from year 2010.
FMI Open data

They already have written java interface but I decided to use Python because I only need to download the data once.
FMI Open Development

I decided to use one wave buoy from Suomenlahti for wave height and water temperature data and two weather stations near it ( Majakka and KalbÄdagrund) for wind speed and air temperature data. The wave buoy is between the two weather stations so I can simply use average data of the weather stations to get the approximate values for wave station area. Using average for wind speed is not actually right because it only works if the wind directions are the same for both stations. But I'm not aiming for 100% accuracy.

The data is fetched with HTTP GET requests and the requests returns data in XML format  I wrote a simple Python script that fetches the data for these three stations and processes it with two XSLT files, one for wave buoy data and one for weather station data.

The data is time-value pairs meaning there is one pair for each measurement.

(This is only a crude example of data structure)
<wave height>
</wave height>

I need two format that data to this:
<time> <temperature> <waveheight>. 

With XSLT it's quite easy to create two for-each-loops and combine two series into one.

So I read the data to bunch of files and downloaded those files to MySql for further analyze of the data.

The source code for my scripts is in GitHub

Analyzing the data

The data at first was three dimensional: air temperature, wind speed and wave height. I would then slice this to different wind speeds and could see if wave height changes with air temperature even when wind speed stays the same. The air temperature was calculated as average of the two weather station temperatures and same with wind speed. But I did not see any correlation.

Then I deciced to use more complex formula for temperature: wave buoy temperature - air temperature. So it uses only the temperature difference of the water and air temperatures and this is how inversion works. Now there was clear correlation when water was colder than the air.

Visualizing the data

Next stage was to visualize that. At first I formatted the data for GnuPlots 3d contour chart.
I created a simple view that I use to access actual data:
create view Contour3d as

Then I created the query that actually outputs the data in GnuPlot format:
    case AreaXY.Y
        when ( select max(Y)+1 from Contour3d ) then ''
        else concat(AreaXY.X, ' ', AreaXY.Y, ' ' , IFNULL(Contour3d.Z,'?'))
INTO OUTFILE 'Wave_correlation.csv'
    (     select RangeX.X as X, RangeY.Y as Y
            (SELECT @row := @row + 1 as X FROM Contour3d , (SELECT @row := ( select min(X) from Contour3d )-1) r where @row <= ( select max(X) from Contour3d )-1) RangeX,
            (SELECT @rowY := @rowY + 1 as Y FROM Contour3d , (SELECT @rowY := ( select min(Y) from Contour3d )-1) r where @rowY <= ( select max(Y) from Contour3d )) RangeY) AreaXY
    LEFT OUTER JOIN Contour3d ON Contour3d.X = AreaXY.X AND Contour3d.Y = AreaXY.Y
order by AreaXY.X, AreaXY.Y;
This works with any kind of XYZ-data. It calculates the boundaries for X&Y, creates the grid and adds '?' to null Z-values.

First chart is 3d contour viewed from 'front'. The wind speed lines are the horizontal lines.

If the wave height is constant and doesn't depend on temperature difference, the horizontal lines would be straight. But they are not. When temperature difference is negative, the wave height is lower. There's also slight correlation that when air temperature is lower than water temperature, waves are lower.

On the bottom you can see the contour lines.

Here's the same from up as a heat map. There's also contour lines. If there is no correlation the contour lines should be straight diagonal.

I imported the same data to LibreOffice and removed those data points that only had 9 or less samples to smooth the lines a bit. Because every higher wind speed means usually also higher waves, it is possible to present this data also as normal 2d charts without losing any information. The lines don't overlap much. Actually this 2d version is more readable than 3d version because there's no perspective projection needed.

In this chart the Y axis is wave height, X axis is temperature difference and the lines are for different wind speed. From this chart you can easily see for instance that when wind speed is 9m/s and temperature difference 0C, the wave height is 1,1m but when the water is -5 C colder than air, the wave height is only 0,55m. 


I'm not a meteorologist. There are many variables in the weather system. The water is of course colder than air in spring and air is colder than water in autumn. But the waves don't start suddenly. They need time to build up. So it might be that for instance in autumn the storms just last longer so waves have more time to build up and the waves are then higher. My goal was not to dig deeper but  to fetch open data from FMI and visualizing that data in 3d and 2d and these goals were met.

Friday, May 23, 2014

How many steps do you have to take to lose weight 1kg?

I have Withings WS-50 scale and Withings Pulse activity tracker. I've now used the scale daily for about a year and Withings Pulse for 8 months. I started wondering would it be possible to see any correlation between the steps I take and weight I lose.

Because I measure my weight almost daily, it's easy to get data about how much weight changes in a day. It's easy to export weight data from Withing web site.

The step data is a little more difficult to get. You cannot export activity data that easily from Withings web site. You have to register a developer account and register your application to get the necessary keys to get access to your data. Then you have to write an application using OAuth and Rest to download the data.

The problem with Withings activity data is there's no way to mark a day's data as NotAvailable meaning you haven't used Pulse that day. So they will be to many days under '0 Steps' day.

So I didn't actually get activity data. That's because I had a better way to measure my steps. I have been working about half year now 50/50. 50% working at home and 50% in the city. The days I work in the city, I know I have taken about 5000 steps because it was always 5000-6000 steps when I had my Pulse with me.

So for every day I calculated the difference of my weight between this day and next day. And for every day I worked in the city, I set the steps as 5000 and for other days I set it as 0 steps.

Here's the results I got:

Steps Weight Difference Kg Count of days
0 1,0442622951 122
5000 -1,3227586207 87

So if I had done 100% work at home, I would be gaining weight 1kg/month and if I had done my work in the city all the time, I would be losing my weight 1,3kg/month. That is a huge difference! There is a lot of noise in the signal but that seems accurate enough. When I was working 100% in the city, I was losing my weight about 2kg/month.

Steps Fat mass difference Kg Count of days
0 6,2678688525 122
5000 -8,1843678161 87

Fat mass difference shows this is not an accurate way to measure weight and activity correlation. 8kg/month fat mass losing? I'm not buying that. This has to have other reasons like dehydrating more after days in the city (more coffee?).

What I would like to have is way to combine all different data sources: Withings activity data, Sport tracker sport data and my own data like this work data to get a common activity data that could be used for more accurate charts like this( this is not real data):

I thought just to be safe to only use those days that are consecutive. If I would be working two days at home, I would only use the second day in statistics and the same with 5000 step - days.

Steps Weight Difference Kg Count of days
0 -0,0031818182 88
5000 -1,1038461538 52
The results are not so impressive anymore but maybe they are more realistic. By doing 5000 steps a day I still get rid of 1,1Kg per month.