Saturday, December 6, 2014

Airwheel - tulevaisuuden kulkumuoto, täällä, tänään!


AirWheel on automaattisesti tasapainottuva pyörä. Samanlainen kuin Segway. Paitsi että Airwheelissä on vain yksi rengas ja koko laite on käytännössä pelkkää pyörää, mukana moottori, jalkatuet, akut ja hieman elektroniikkaa. Koko paketti on minimalistin ihanne.




Kun näin laitteesta videoita ensimmäistä kertaa, oli samalla kertaa myyty. Tämä tulee mullistamaan ainakin minun työmatkaliikkumiseni.

Keksintö on itse asiassa tehty jo pari vuotta sitten amerikkalaisen Solowheel yrityksen toimesta. Solowheel vain markkinoi omaa pyöräänsä yli 1000e hinnalla joten ei mikään ihme ettei se ole saanut ilmaa siipiensä alle. Nyt kuitenkin on tullut kiinalainen kilpailija Airwheel joka on puolittanut kulkuneuvon hinnan ja tällöin se on saanut jo huomattavan paljon suuremman potentiaalisen asiakaskunnan.

Kyseessä on siis yksipyöräinen mikä tarkoittaa että tasapainotaitoa tarvitaan enemmän kuin polkupyörässä. Kaikille jo pyöräilyn jo oppineille "aikuisille" ongelma on tietenkin siinä että pitää nöyrtyä opettelemaan pyörälläajo uudestaan. En usko että se ihan kaikille sopiikaan. Toisaalta kuinka harva ei koskaan opi ajamaan polkupyörällä? Minulla taustalla on esim. leijasurffausta joten tasapainon löytymisen tiesin olevan vain ajan kysymys. Ei kuitenkaan olla vielä lähelläkään sitä sujuvuutta mitä löytyy esimerkiksi Youtubesta löytyvistä videoista. Tässä kaksi suosikkiani:




Maksimivauhti on 18km/h vaikka normaali matkanopeus on alle 12km/h. Se on kuitenkin yli tuplasti enemmän kuin kävelynopeus. Tämä muuttaa kaupunkikulkemisen "maantiedettä" kun kaikki bussinvaihtojen väliset matkat voi tehdä tällä ja jättää jopa bussit kokonaan väliin. Airwheelin paino on n. 10kg joten kovin pitkiä matkoja sitä ei viitsi/jaksa kanniskella vaikka netin videoissa neitoset sitä kevyenoloisesti nostelevatkin. Uskonkin että tulevaisuudessa kun käyttäjäkunta kasvaa, painoa saadaan vielä hilattua alemmas. Löytyy kuitenkin jo käyttäjien itse-tekemiä "kantokahvoja" joilla Airwheeliä voidaan vetää perässä tai työntää edessä kuin matkalaukkua. Kehitys on vasta alussa.

Suomessa Airwheeliä markkinoi esimerkiksi teinitähti Robin. Toivoisin kumminkin ettei Airwheel identifiotuisi pelkästään nuorison temppulaitteeksi. Silti Robin osaa omissa videoissaan käyttää Airwheeliä paljon näppärämmin kuin minä.

Minun laitteessani on 170Wh:n akku jolla pääsee n. 23km:n matkan. Kun matkanopeus on 12km/h niin se tarkoittaa n. kahden tunnin käyttöaikaa ja n. 90W:n kulutusta(arvio). Kun sähköauto vie n. 23kW niin tämä 90W:n kulutuksella on varmaan se kaikkein tehokkain moottorikulkuneuvo.

Akkupaketti ei ole vaihdettava ts. mitään vaihtoakkuja ei ole myynnissä mutta toivottavasti akut voi vaihtaa myöhemmin ruuvimeisseliä käyttäen. Akkujen pitäisi kestää 1800 latausta joten päivittäisellä käytöllä kysymys tulee eteen vasta viiden vuoden päästä. Kilometreinä tuo tekee 40 000 kilometriä.

Tietenkään tätä ei ole vielä hyväksytty tieliikennekäyttöön. Suomen lainsäätäjät kun ovat näissä asioissa niin nopealiikkeisiä ja turvaallisuustietoisia ettei Suomessa saa vielä käyttää edes Segwaytä kun se on niin vaarallinen. Mainittakoon tässä että esim. Barcelonassa ja Berliinissä järjestetään turisteille Segwayllä turistikierroksia ts. turisteille pieni koulutus ja sitten normaalin liikenteen sekaan jonoon ajamaan. Ja Suomessa esim. poliisi ei saa ajaa omalla Segwayllaan koska se on laitonta...

Ehkä tässä kuitenkin on toivetta lainsäädännön muutoksesta, kun Liikenne -ja viestintäministeriössä on asiaa taas pohdittu:
LVM: Kevytajoneuvot: Yhteenveto työpajoista ja pienpalavereista
Iltasanomat : Uuteen tieliikennelakiin tulossa koko joukko aiemmin tuntemattomia ajoneuvoja

Itse rinnastaisin tuon polkupyörään painon perusteella. Itsekseen se ei lähde ajelemaan eikä painoa tosiaan ole enempää kuin 10kg.

Sain oman laitteeni viime viikon alussa ja ensimmäisen opettelu oli jo keskiviikkona. Minulla ei ole apupyöriä ja päätellen oppimisen nopeudesta ne ovatkin aika turhia. Mukana oli hihna jolla voin tukea Airwheeliä liikkeessä mistä oli hyötyä alussa.

Ensimmäisiä opetteluita: ota tukea jostain ja nouse pyörän päälle. Tässä vaiheessa on paras että pyrit pistämään jalat jalustojen keskelle. Pitäen tuesta kiinni liikuttele pyörää edestakaisin jotta saat tuntuman siitä miten kiihdytys, jarrutus ja takaperin ajo sujuu. Muista että nämä pitää mennä lihasmuistiin, joten älä katso pyörää vaan katso eteenpäin ja mieti enemmän sitä miltä pyörä tuntuu. Silmistäsi ei oikeastaan ole tässä hyötyä, tärkeintä on lihasmuisti ja tasapainon oppiminen. Voit toisella kädellä ottaa tukea ja toisella pitää pyörän tukihihnaa.

Ilman tukea pyörälle nousu oli sitten niitä vaikeimpia opeteltavia kun alussa oli hyvin tarkkaa että sain jalat oikeaan kohtaan. Vaikka neuvottiinkin että on paras että jalat ovat tuen keskellä niin minusta oli parempi että olin hieman etupainoinen ts. päkiä oli tuen reunalla. Tällöin jo heti pyörälle noustessa se hieman lähti menemään eteenpäin ja hyrrävoima auttoi pystyssä pysymisessä. Sitten kun nousun oppii niin kaikki tulee jo selkäytimestä eikä näitä tarvitse erikseen miettiä ja kiihdytys toimii kuin ajatus. Opetteluvaiheessa viivettä oli vain liikaa pyörälle nousu - eteenpäin lähtö. Tästä päivästä jäi myös kunnon mustelmat kumpaankin jalkaan nousuharjoituksista. Keskiviikkona meni opetteluun n. 2 tuntia ja en päässyt kuin maksimissaan ehkä viitisen metriä kerrallaan.

Torstaina sitten tajusin vasta puolivälissä kahden tunnin treeniäni että itse asiassa tässä vaiheessa tukihihna vain haittaa asiaa kun en voi käyttää kumpaakin kättä tasapainotteluun joten jätin hihnan pois ja kummatkin kädet lentokoneasentoon. Tämän jälkeen pääsinkin heti 50 metriä eli parkkipaikan päästä päähän. Torstain toinen puolisko menikin sitten pitempiin ajoihin ja kääntymisen opetteluun. Otin myös silmälasit pois mikä helpotti sitä että pystyi keskittymään ajamaan "fiiliksellä". Näen joka tapauksessa muutoin riittävän hyvin mutta jokaisen pikkukiven ja oksan näkeminen ei muuta kuin haitannut keskittymistä. Vasta nyt loppui akku ensimmäisen kerran kesken ts. opetteluvaiheessa se kesti nelisen tuntia.

Perjantaina sitten kun sain taas palautettua edellisen päivän opit muistista, ajoinkin melkein koko kahden tunnin. Keskeytyksiä tuli varmaan vain kymmenisen kertaa. Opettelussa oli käsien käytön vähentäminen, käännökset ja pujottelu. Pyrkimys oli ajaa mahdollisimman vähän suoraan koska siinä oppii vähiten. Muun liikenteen eli jalankulkijoiden seassa en pysty vielä ajamaan joten parkkipaikkaharjoittelua ja hiljaisempia pyöräteitä. Ja tosiaan, pyöräteillä ajetaan yleensä suoraan joten mieluummin pysyin parkkipaikalla opettelemassa kääntymisiä.

Talvi tulee joten yksipyöräisen käyttö jäänee vähemmälle mutta etelän talvet eivät ole pitkiä joten eiköhän keväällä Airwheel ole hyvinkin pian taas käytössä.

Sunday, November 30, 2014

About benchmarking Hadoop systems and scalability

I've been reading a lot of benchmarks for Hadoop technologies lately and these benchmarks that software companies make should always be taken with a grain of salt. Every new version or software is 10-100 times faster than the others.


IBM did a benchmark with Big SQL 3.0 against Cloudera Impala and Hortonworks Hive. The results were very good and I'm not claiming the results are false. But there was one claim that caught my attention.

There were some charts about scalability when the same 99 test queries were run with 1 user vs. 4 users. The 4 concurrent users test took only 1,8 times longer than 1 user test and it was said it is a sign of good scalability.

Blistering Fast SQL Access to Hadoop using IBM BigInsights 3.0 with Big SQL 3.0

Of particular note is the fact that 4 concurrent query streams (and therefore 4 times more queries) only takes 1.8x longer than a single query stream at 30TB. Once again highlighting Big SQL’s impressive multi-user scalability – this time at 30TB.


Serialized vs. parallelized execution 

 

With parallel systems it is essential that the execution can be parallelized as evenly between all the nodes as possible. If in test system ( 17 nodes in IBM's test) only one node is used 100% and others are idle, that means the execution is serialized.

Even then the execution is serialized if the it uses multiple nodes, but only one node is used at a time. I've seen software that should be multi-threaded but it in fact behaves single-threaded, using many processors but only one at a time.



Here is an example of 20 second query that uses all 4 nodes but actually it is only using one node at a time. The execution in the next node does not start before the earlier node has finished. The execution is serialized.



Here is perfectly parallelized execution. It uses 20 seconds of CPU as the other one, but now it uses all nodes in parallel and the whole execution only lasts for 5 seconds. Normally these kind of executions exist only in computer labs. These have not been seen in wild. If you see, contact local authorities.

What can be parallelized?

 


In a Hadoop system the primary resources that can executed in parallel are CPU, memory, disk and network. Each of these resources is also finite and can be overloaded. Execution can also be parallel in that way that one thread uses CPU, one thread disk and one thread network without affecting each others performance.

When a resource is overloaded it might simply make every execution slower in a linear way but a resource might also be so overloaded that it cannot execute any more requests like what happened to Impala and Hive in the test with memory running out.

How parallel was Hadoops execution in test?


If the test query is evenly parallelized, then every node and every resource is used 100%. If you run 4 concurrent tests then if it's is evenly parallelized,  the minimal time it takes to execute is four times longer. If 4 concurrent tests take less time than that, then it wasn't evenly parallelized in the first place. Remember that this applies to every resource in every node: CPU should be used 100% and every disk should by used 100%.

If I'm doing a benchmark, of course I select that amount of concurrent users that gives the results that appear to show best parallel execution.

In an execution there are parts that are serialized and parts that are parallelized. Lets try to calculate how much there is serialized execution in this Hadoop test. Lets make highly simplified model.

S is the execution time for serialized part and P is the execution time for parallelized part. Single_time is test execution times for single user test and parallel_time is the time of one execution in multiple concurrent users time. Threads is the amount of concurrent users and nodes amount of nodes.

Lets make assumption in our model that the serialized parts always get one whole node because that would be the case if we want to show results that appear as best parallel execution. The parallel part has to divide nodes between concurrent users.

single_time = S + P/nodes
parallel_time = S + threads*P/nodes

Let do some math:
single_time = S + P/nodes
single_time - S = P/nodes
nodes(single_time - S) = P

parallel_time = S + threads*nodes(single_time - S)/nodes
parallel_time = S + threads*(single_time - S)
parallel_time = S + threads*single_time - threads*S
parallel_time - threads*single_time = S - threads*S
parallel_time - threads*single_time = (1 - threads)*S
(parallel_time - threads*single_time)/(1 - threads) = S
S = (parallel_time - threads*single_time)/(1 - threads)

P = nodes(single_time - (parallel_time - threads*single_time)/(1 - threads))

In IBM's test the 4 user test was 1,8 times longer then single user test so:
single_time = 1
parallel_time = 1,8
threads = 4
nodes = 17
=>
S = 0,733....
P = 4,533...
Whole execution time S+P = 5,266...
S% (serialized part percentage) = 13,9%


We can calculate how these figures change if we add or substract nodes:
          Nodes         S         P         S+P         S%
3 0,7333 0,8000 1,5333 47,8%
4 0,7333 1,0667 1,8000 40,7%
5 0,7333 1,3333 2,0667 35,5%
6 0,7333 1,6000 2,3333 31,4%
7 0,7333 1,8667 2,6000 28,2%
8 0,7333 2,1333 2,8667 25,6%
9 0,7333 2,4000 3,1333 23,4%
10 0,7333 2,6667 3,4000 21,6%
11 0,7333 2,9333 3,6667 20,0%
12 0,7333 3,2000 3,9333 18,6%
13 0,7333 3,4667 4,2000 17,5%
14 0,7333 3,7333 4,4667 16,4%
15 0,7333 4,0000 4,7333 15,5%
16 0,7333 4,2667 5,0000 14,7%
17 0,7333 4,5333 5,2667 13,9%
18 0,7333 4,8000 5,5333 13,3%
19 0,7333 5,0667 5,8000 12,6%
20 0,7333 5,3333 6,0667 12,1%
21 0,7333 5,6000 6,3333 11,6%
22 0,7333 5,8667 6,6000 11,1%
23 0,7333 6,1333 6,8667 10,7%
24 0,7333 6,4000 7,1333 10,3%
25 0,7333 6,6667 7,4000 9,9%
26 0,7333 6,9333 7,6667 9,6%

We can see that serialized part of time stays the same so amount of nodes does not affect it. The test had at least 17 nodes and as I said earlier the actual "node" count is higher because it should be the amount of individual resources(CPU & disk in each node) . But as we can see from the calculation, the S% does not change that much when node count is higher. (And you have to also understand this is only a simplified model )

The parallelized execution time P is the whole summed execution time of all nodes used. So actually the time it takes to execute the parallelized part is constant 0,266... I'm not saying it does not matter how many nodes you use in your test. I'm saying to get these kind of results (one thread 1, 4 threads 1,8x) it does not matter how many nodes you had in your test. It only matters how the execution was parallelized.

We can also see than when there are 17 nodes, the serialized part takes 73% of the execution time and parallized part 27%. 

How does it scale then? 


Let's make a simple example where the original test only had 4 nodes.

This is how a single thread executes:



First it executes the parallel part in 4 nodes and then continues with serialized part in one node. The reality is of course much more complex. As you can see, when the serialized part starts, three nodes are idle.

Here is what happens when 4 threads are run in 4 node system.




The serialized part executes as single thread test but parallelized part has to divide nodes between all threads. Then 4 thread execution is about 1,8 times slower than single thread execution. No nodes are idle. This is how 4 thread execution is only 1,8 times slower:in the single thread execution there are idle resources because it is not perfectly parallelized.

What happens when we add 5. thread?


We start to see normal behaviour in scaling. There are no idle nodes to utilize any more.

Conclusion


I don't have the test results of individual queries or data about resource consumption So I had to try to mathematically make a model about what is happening. There might also be other reasons like disk caches because all 4 threads execute the queries in the same order (actually that was not said in paper how it was run).

But I'm sure one of the reasons is also serialization. It is not a good thing to have serialized parts in executions because if you add more nodes to have faster executions, only the parallelized part gets faster. If it's true that 73% of execution time in test was serialized that is the part you will never get rid of by simple adding more nodes. This is why it's is good to have a proper benchmarking when you build your Hadoop system.

Tuesday, November 11, 2014

IoC ( Internet of Cigars) - enabled humidor!

The latest trend: IoC or Internet of Cigars is here! :)

I tried for a long time to keep humidity in my cigar humidor at constant level. No luck. I have humidity beads from Heartfelt Industries designed to keep 65% humidity but still there was too much fluctuation in humidity, one loss of all cigars because of mold and too much work to check the humidity levels often enough.

That was before my humidor was connected to the Internet of Cigars.

The sensor

My humidor has Zwave humidity sensor Everspring ST814 that sends report to my home automation system every time humidity changes 5% and temperature by 0,1C. I would like to have a sensor that reports humidity even more often but the market is small.

Home automation system itself doesn't do anything for data because it has limited capabilities for graphing the historical values so I've configured it that is only receives the Zwave reports. Home Automation software I use (OSA) has also RESTful interface so it's easy to make queries of current humidity in humidor.

The monitoring and graphing


For graphing and alerts I use Cacti, widely-used monitoring and graphing tool. Cacti is open-source which means it's not the user-friendliest software in the world but like many OS software it is powerful and versatile. It is easy to write scripts to Cacti that then acts as sensors. The Cacti is written in Perl so the native language for monitoring scripts is also Perl. The Cacti script for Zwave humidity sensor simply fetches the RESTful data in JSON and reads the temperature and humidity values to Cacti. With Perl it's only a 10-20 lines of code.

The database 


Cacti uses RRD database for sensor data and RRD stands for Round Robin Database. RRD saves values in pre-created constant-sized database files and when it reaches the end of file it starts over in Round Robin fashion. There can be many files in that way that one file contains data in week level, one in month level and one in year level each with different temporal resolutions.



Here's a hourly view.


Daily view, same straight line so let's skip the Weekly and Monthly views and go straight to Yearly view.


The notifications


Cacti has a plugin architecture and it being open source, it has lots of plugins. Plugin I use to alert me is Threshold. When the humidor humidity is too high(>75%) or too low(<60%), the Threshold plugin sends me an email every four hours. As you can see from the yearly view I have to add some water to humidor once in a month. I could tighten the warning levels a little bit but this is not too much work for a lazy smoker I am. The alerts are send with sendmail so some additional configuration was needed that Cygwin environment was able to send mails via SSMTP.


 
My Internet of Cigars keeps a constant watch of my cigars well-being and gives orders to water boy (me) to refill the beads with water. Now I can enjoy my cigars with no stress and wait for IoC to appear in Gartner reports :)


Sunday, November 9, 2014

Reason why Donetsk People's Republic's elections held in 2.11.2014 are fraud


First you should read the blog post I use as a reference.

Elections in Donets - anatomy of a fraud.
I think this needs more publicity. I try to explain how this kind of results of elections in East Ukraine are impossible.

First the official results:
Zaharchenko     : 765 340 / 969 644
Kofman             : 111 024 / 969 644
Sivokonenko      :  93 280 / 969 644
Discarded votes : 43 039 / (969 644+43 039)

Everything seems to be ok?

Then we calculate the percentages:
Zaharchenko     : 765 340 / 969 644 = 78,92999905119820000%
Kofman             : 111 024 / 969 644 = 11,44997545490920000%
Sivokonenko      :  93 280 / 969 644 = 9,62002549389260000%
Discarded votes :  43 039 / (969 644+43039) = 4,24999728444143000%

The problem with these figures is that the first two decimals are ok, but then at least the next two decimals are either 00 or 99 which means they are very close to two-decimal numbers. All of them.

Lets take Zaharchenko's votes for further examination:
Zaharchenko: 765 340 / 969 644 = 78,92999905119820000%
That is really close to 78,93%
In fact it is not possible to get percentage to 78,93% because you would not get whole votes:
969 644 * 78,93% = 765340,0092000 votes
So if you first "invented" the two decimal percentage as voting result and then calculate the vote count, you would have to truncate to 765340.

There were 969 644 accepted votes. That means there are 969 644 possible results for Zaharchenko and probability of any result is 1/969644.

There are 10000 possible percentages with two decimals. Because the accurate two decimals results are not possible and it's almost always between two votes, lets take the both upper and lower vote counts. So possible vote counts for two decimal percentage are ~20000.

The probability that Zaharchenko's vote count is one vote near the two decimal percentage is 20000/969644 ~ 1/50. That is 2% probability. That is not impossible, that could happen. It means that if you held elections anywhere in the world and you pick 50 candidates, at least one of them is propable to have two decimal percentage.

Let's then get the two decimal percentages for all the candidates:
Zaharchenko     : 765 340 / 969 644 = 78,93%
Kofman             : 111 024 / 969 644 = 11,45%
Sivokonenko     :   93 280 / 969 644 = 9,62%
Discarded votes :  43 039 / (969 644+43 039) = 4,25%

It's here when things get highly improbable.
Zaharchenko's probability 1/50
Kofman's probability 1/50
Sivokonenko probability 1/50
Discarded votes probability 1/50.

Because sum of candidates votes is 100% it is not possible that two candidates have two decimal percentages and the third to have more decimals. So we only use probabilities of the candidates because the third one's probability is "tied" to two first ones probability and therefore his probability is 1/1 meaning it always happens. The probability that the three candidates all have two decimal percentages is therefore 1/50 * 1/50 * 1 = 1/2500.

Then we add the discarded votes and the probability that all the percentages have two decimals is 1/2500 * 1/50 = 1/125 000 or 0,0008% probability . That's the smoking gun. That means DNR should have 125 000 elections and get one result like this. Of course, it is still possible. At least in Donetsk People's Republic. 

PS. The elections held in May 2014 in DNR and LNR had only one decimal percentages.

Monday, November 3, 2014

Creating manual standby in Oracle 12c SE with PDB/CDB Multitenant architecture

Creating a manual standby with Oracle 12c SE is a cost-effective way to create entry level fault tolerance. With manual standby the worst case scenario is that you lose updates of a couple of last minutes in case of server failure. This depends of how you have configured your database to do log switches and archive log copying to standby server. If you do log switches every 5 minutes and rsync every minute at worst you can loose 6 minutes worth of updates. Manual standby also has to be activated manually. There will be no automatic failover.


With Standard Edition you are only allowed to create manual standby, not Managed Standby. This means you have to copy the archive logs manually and do the recovery manually. If you keep also the standby mounted but not in open mode, you don't have to have a license for a standby server.
The Care and Feeding of a Standby Database

"The Managed Standby normally does not require a separate Oracle License because it is not open for use.  Normally only one database is active at a time.  Notice the use of the word normally.  If you use your standby for reporting or run it open-read only (11g) you will need to licenses the database."
"To create a Managed Standby database, you must be using the Enterprise Edition of the database.  If you are using the Standard Edition of the database, you can still create a manual standby database, but not a Managed Standby."

With Standard Edition  you also can create Pluggable Databases but are only allowed for one PDB per CDB. You will not get all benefits of Multitenant architecture but you will be able to upgrade the database with little less downtime.
Multitenant with Only One Pluggable Database: An Upgrade Option to Consider

"In using a single PDB, the features of plugging and unplugging are available. Being able to do this provides another patching option in that you can unplug the PDB and plug it into an already-patched CDB. This can provide a patching scenario with less downtime and stronger testing plans."
Because the redo logs and archive logs are in the CDB level, the standby database has to be copy of the whole CDB database also. There are no separate redo logs for each PDB so you cannot replicate a single PDB only.


I suggest using at least version 12.1.0.2 because that version contains a feature to automatically open PDB's to open state when CDB is restarted.
12.1.0.2 New Features: PDB State Management Across CDB Restart

I prefer to use a separate staging area for creating a standby database. You can then first create a copy of primary database to staging area and as a separate step create a new standby database from that copy. 

Step 1: Prepare primary database

 

Create Environment file

 

It will be easier to create scripts if you have all the variables of your system in one file. When you then run your scripts, you simply run the environment file before that.
Environment variables I have used:

#ORACLE_SID of primary CDB database
PRIMARY_SID=TEST

# Directory where Standby files are. Same directory in Standby and Primary server.
STANDBYLOGS =/u02/Standby

#ORACLE_SID of standby CDB database
STANDBY_SID=TEST

#Root dir of Oracle database files (for all databases)
ORACLE_DATA_DIR=/u02/oradata

#Root dir of Oracle admin files. 
ORACLE_ADMIN_DIR=/u01/app/oracle/admin

# Primary FRA directory
PRIMARY_FRA=/u02/fast_recovery_area

Mount Standby directory to Primary database. 

 

This directory will be used for transferring copy of data files and archive logs. It is easiest to create a "Standby" directory to Standby server and mount that directory to same name and path in Primary server. Just remember not to use a directory in Primary server because you will then lose that data in server failure.

Remember to check that 'oracle' user has the same user id in both servers so that access permissions are the same for all the directories and files in the shared Standby-directory.

Set Primary Database to Archive Log mode

 

export ORACLE_SID=$PRIMARY_SID
sqlplus / as sysdba
shutdown immediate;
startup mount;
alter database archivelog;
alter database open;

 

Enable force logging

 

alter database force logging;

 

Create and edit standby init.ora


#Get pfile from primary database 
export ORACLE_SID=$PRIMARY_SID
sqlplus / as sysdbaCREATE PFILE='$STANDBYLOGS/init$PRIMARY_SID.Primary.ora' FROM SPFILE;
quit;
 
cp  $STANDBYLOGS/init$PRIMARY_SID.Primary.ora
$STANDBYLOGS/init$STANDBY_SID.Standby.ora

Edit init$STANDBY_SID.Standby.ora

Change controlfiles to standby controlfile location. Remember to change directories, you cannot use Unix variables in here. 
*.control_files='/u02/oradata/TEST/controlfile/standby1.ctl', '/u02/oradata/TEST/controlfile/standby2.ctl','/u02/oradata/TEST/controlfile/standby3.ctl'

Change FRA  to standby's FRA
*.db_recovery_file_dest='/u02/standby/fast_recovery_area'

It's easiest (IMHO) to use listener configured in tnsnames.ora. (TEST is the database sid.)
*.local_listener='LISTENER_TEST'

Configure listener alias in tnsnames.ora in primary and standby servers. Both have their own hostname in HOST setting(servername.domainname) . TEST is the database sid. This way you don't have to set hostname's in init.ora's.
LISTENER_TEST =
  (ADDRESS = (PROTOCOL = TCP)(HOST = servername.domainname)(PORT = 1521))

 

No database autostart in primary server

 

Please check there is no autostart of production database in primary server. If there's is a failure in primary server and standby is activated, there is no quick turning back to primary database. Primary database has to be recreated from currently active standby database meaning you have to stop standby, take backup and move that backup to primary database. Therefore if somehow primary server is back online, and primary database is autostarted, the users start to use it instead of standby and you end up with two databases used simultaneously that are then not in sync.

 

Step 2: Create copy of primary database

 


These commands run in primary server will create a file copy of primary database online. No need for shutdown and downtime. But this is a backup so I advice not to run this at a busy daytime hours.

#Switch log file and start backup
sqlplus / as sysdba << END
alter system archive log current;
alter system switch logfile;
alter database begin backup;
quit;
END
 

# Take backup of datafiles
# This takes a backup of all datafiles in CDB and PDBrm -rf $STANDBYLOGS$ORACLE_DATA_DIR/$PRIMARY_SID
mkdir -p $STANDBYLOGS$ORACLE_DATA_DIR/$PRIMARY_SID
cp -r $DATA_DIR/* $STANDBYLOGS$ORACLE_DATA_DIR/$PRIMARY_SID
rm -rd $STANDBYLOGS$ORACLE_DATA_DIR/$PRIMARY_SID/onlinelog
rm -rf $STANDBYLOGS
$ORACLE_DATA_DIR/$PRIMARY_SID/controlfile

# Stop backup and create standby control file
sqlplus / as sysdba << END
alter database end backup;
 
rm -rf $STANDBYLOGS/standby_$PRIMARY_SID.ctl
ALTER DATABASE CREATE STANDBY CONTROLFILE AS '$STANDBYLOGS/standby_$PRIMARY_SID.ctl';
# Create also pfile. We don't need it but it's good to have a copy in case you need to check quickly parameters. 

CREATE PFILE='$STANDBYLOGS/init$PRIMARY_SID.Primary.ora' FROM SPFILE;
alter system archive log current;
alter system switch logfile;

quit;
END

 

Step 3:Create standby database from primary database copy

 

These commands are run in standby server and they create the standby database.

If you activate standby, it creates an automatic backup. If you after that create the standby database again, you cannot start recovery when RMAN registers that old backup. The standby database has then different incarnation than primary database.Therefore when we recreate the standby, we have to remove any earlier backups from standby's FRA.

#Shutdown standby database if it exists 
export ORACLE_SID=$STANDBY_SID
lsnrctl stop listener
sqlplus / as sysdba << END
shutdown immediate;
quit;
END
 

# Create directories and copy files
rm -rf $
ORACLE_DATA_DIR/$STANDBY_SID
mkdir -p $
ORACLE_DATA_DIR/$STANDBY_SID
cp -r $STANDBYLOGS$
ORACLE_DATA_DIR/$PRIMARY_SID/* $ORACLE_DATA_DIR/$STANDBY_SID
mkdir $
ORACLE_DATA_DIR/$STANDBY_SID/onlinelog

# Copy control files
mkdir -p $
ORACLE_DATA_DIR/$STANDBY_SID/controlfile
cp $STANDBYLOGS/standby_$1.ctl $
ORACLE_DATA_DIR/$STANDBY_SID/controlfile/standby1.ctl
cp $STANDBYLOGS/standby_$1.ctl $
ORACLE_DATA_DIR/$STANDBY_SID/controlfile/standby2.ctl
cp $STANDBYLOGS/standby_$1.ctl $
ORACLE_DATA_DIR/$STANDBY_SID/controlfile/standby3.ctl
#
mkdir -p $ADMIN_DIR/$STANDBY_SID/pfile
# DB Parameter file
cp $STANDBYLOGS/init$STANDBY_SID.Standby.ora $ADMIN_DIR/$STANDBY_SID/pfile


# Remove earlier standby backups  

rm -rf $STANDBYLOGS/fast_recovery_area/$STANDBY_SID/autobackuprm -rf $STANDBYLOGS/fast_recovery_area/$STANDBY_SID/onlinelog

# Avaa kanta
sqlplus / as sysdba << END
STARTUP NOMOUNT pfile='$ADMIN_DIR/$STANDBY_SID/pfile/init$STANDBY_SID.Standby.ora';
create spfile from pfile='$ADMIN_DIR/$STANDBY_SID/pfile/init$STANDBY_SID.Standby.ora';
ALTER DATABASE MOUNT STANDBY DATABASE;
ALTER DATABASE RECOVER AUTOMATIC STANDBY DATABASE UNTIL CANCEL;
alter database recover cancel;
END

 

Step 4:Keeping up with the primary database


Copy archive logs to standby

 

In primary server you have to run in cron these commands to replicate archive logs to shared standby directory.

mkdir -p $STANDBYLOGS/fast_recovery_area/$STANDBY_SID/archivelog
rsync -aqz --delete $PRIMARY_FRA/
$PRIMARY_SID/archivelog/ $STANDBYLOGS/fast_recovery_area/$STANDBY_SID/archivelog

 

Apply Archive logs to standby

 

In standby server you have to run redo apply in cron.

# Register the archive logs found in FRA
export ORACLE_SID=$STANDBY_SID
rman << EOF
connect target /
crosscheck archivelog all;
delete noprompt expired archivelog all;
quit;
EOF
 

# Apply recover to standby
sqlplus / as sysdba > /dev/null << END
ALTER DATABASE RECOVER AUTOMATIC STANDBY DATABASE UNTIL CANCEL;
alter database recover cancel;
quit;


 

Delete used archive logs

 

You also have to delete old archive logs from primary database but that depends how you backup the primary database. I prefer to keep archive logs for a week. That way you can always create a standby database again from a week old copy and just apply the archive logs from a week. It will take a long time, but at least you dont't have to do anything heavy in primary server.

 

Step 5. Activating the standby. 


I'm not posting complete instructions of failover to standby because in real world situations at first you try to get primary back online first. Then you try to get as much archive logs out of primary server as possible and then you start to activate standby server.

In testing you can activate the standby this way:
sqlplus / AS SYSDBA << END
ALTER DATABASE ACTIVATE STANDBY DATABASE;
shutdown immediate
startup;
END

 

Conclusion

 

Using multitenant architecture with manual standby needed only minor changes in scripts, mainly in the location of datafiles. In Non-CDB database all the datafiles normally are in one /datafile-directory. In multitenant architecture the CDB datafiles are in /datafile-directory and each PDB(in this case only one) has it's own directory where /datafile -subdirectory resides.

Tuesday, October 28, 2014

Prolimit Predator Kite Harness

Like I said in Good Stuff:Kite harness Prolimit Aaron Hadlow Special I bought KiteWaist Pro to replace the old war horse. But that wasn't flexible enough for my liking and I decided I would use it only for snowkiting and maybe with drysuit in cold.

For summer kiting I then bought Prolimit Predator and have now used it for one summer.


It feels great, gives me the same flexibility than the Aaron Hadlow Special and quality seems to be the same high level. I'm pretty sure this thing will last as long as the Aaron Hadlown Special did.

Saturday, September 27, 2014

Qi charging : it just works

I bought Samsung S4 Active last spring and I didn't do my homework properly. S4 Active does not have any official Qi charging covers.

So i had to go unofficial. It is possible to insert Qi tag inside S4 Active's backcover but the tag needs to be very thin and at the same time you loose some of its water tightness. And if you loose only some of it, you loose all of it. It is not water tight anymore.So, it's you choice...

This one is slim enough to fit inside S4 Active:
S4 SlimPWRcard

It seems Qi market is still small. I had to order some parts from Germany, some from USA and some from Korea.

As a home charging station I bought LG WCP-400. It took some time to find a dealer because nobody in Europe didn't have it anymore and I had to order that one from USA and not all dealer's deliver to Europe.


I basically ordered this one for nice design and it's easy to read your phone's screen when it's tilted.

For car there was not a lot of choices. LG has a smart phone holder SMART FIT2 (TCD-C800P) for car that can be installed with charging station LG WCP-300.


In the picture above the smart phone holder has a white LG WCP-300 inserted inside. The car mount I had to order via Ebay from Korea.

Of course it was difficult to find original LG WCP-300 also so I had to order a china-copy, actually two. There's a lot of them in Amazon and I think they are all from the same factory. They fit nicely inside the car mount.

Because smart phone's of today need daily dose of electricity using wireless charging removes all that inconvenience. You just place the phone into it's stand and it starts charging.