Maybe Some Useful Information On Distributed Statistics Maintenance Options

[kkith]

I performed a fresh openx 2.8 install on our cluster that runs openx.

I then performed eveything here to get distributed statistics working...
http://www.openx.org/en/docs/2.8/adminguid...uted+statistics

Created a test campaign/banner and test publisher. Created a test web page, inserted invocation code and viewed the banner.

I waited a couple of hours and noticed no statistics were gathered...not good.

I noticed a few things in var/debug.log
"Apr 21 04:30:01 +0000 OX [ info] Distributed stats disabled, not running Maintenance Distributed Engine"

I then checked the file scripts/maintenance/maintenance-distributed.php which led me to /lib/OX/Maintenance/Distributed.php and found this on line 58 (using VIM)...

"if (empty($GLOBALS['_MAX']['CONF']['lb']['enabled'])) {"

Apparently you have to have the [lb] enabled on your delivery boxes. Nowhere in the instructions for distributed-statistics does it mentioned it having to be enabled on your delivery boxes. It did in the 2.6 instructions but not the 2.8.

So I filled in the [lb] section on all delivery boxes.

Then I ran the scripts/maintenance/maintenance-distributed on the CLI and got this in the debug.log...

CODE

707 Apr 21 05:49:05 +0000 OX [    error]  PEAR :: MDB2 Error: Array : _doQuery: [Error message: Could not execute statement]
708 [Last executed query: SET SQL_LOG_BIN = 0]
709 [Native code: 1227]
710 [Native message: Access denied; you need the SUPER privilege for this operation]
711
712 Apr 21 05:49:05 +0000 OX [  warning]  htmlspecialchars() expects parameter 1 to be string, array given
713 Apr 21 05:49:05 +0000 OX [emergency]  -111 :: Unable to disable the bin log - will not insert stats.

So I then went to the main db and gave the SUPER privilege to the specific user. and re-ran the script by hand and got this...

CODE

752 Apr 21 06:15:01 +0000 OX [     info]  Running Maintenance Distributed Engine
753 Apr 21 06:15:01 +0000 OX [     info]    - Processing the ox_data_bkt_c table for data with operation interval start equal to or before 2009-04-21 05:00:00     UTC
754 Apr 21 06:15:01 +0000 OX [     info]    - Processing the ox_data_bkt_a table for data with operation interval start equal to or before 2009-04-21 05:00:00     UTC
755 Apr 21 06:15:01 +0000 OX [     info]      - The ox_data_bkt_a table is a raw data table. Data logged in real-time, not operation intervals.
756 Apr 21 06:15:01 +0000 OX [     info]      - Accordingly, processing of the ox_data_bkt_a table will be performed based on data that has a logged date equa    l to
757 Apr 21 06:15:01 +0000 OX [     info]        or before 2009-04-21 05:59:59 UTC
758 Apr 21 06:15:01 +0000 OX [     info]      - The ox_data_bkt_a table is a raw data table. Data logged in real-time, not operation intervals.
759 Apr 21 06:15:01 +0000 OX [     info]      - Accordingly, pruning of the ox_data_bkt_a table will be performed based on data that has a logged date equal t    o
760 Apr 21 06:15:01 +0000 OX [     info]        or before 2009-04-21 05:59:59 UTC
761 Apr 21 06:15:01 +0000 OX [     info]    - Processing the ox_data_bkt_a_var table for data with operation interval start equal to or before 2009-04-21 05:0    0:00 UTC
762 Apr 21 06:15:01 +0000 OX [     info]      - The ox_data_bkt_a_var table is a raw data table. Data logged in real-time, not operation intervals.
763 Apr 21 06:15:01 +0000 OX [     info]      - Accordingly, processing of the ox_data_bkt_a_var table will be performed based on data that has a logged date     equal to
764 Apr 21 06:15:01 +0000 OX [     info]        or before 2009-04-21 05:59:59 UTC
765 Apr 21 06:15:01 +0000 OX [     info]      - The ox_data_bkt_a_var table is a raw data table. Data logged in real-time, not operation intervals.
766 Apr 21 06:15:01 +0000 OX [     info]      - Accordingly, pruning of the ox_data_bkt_a_var table will be performed based on data that has a logged date equ    al to
767 Apr 21 06:15:01 +0000 OX [     info]        or before 2009-04-21 05:59:59 UTC
768 Apr 21 06:15:01 +0000 OX [     info]    - Processing the ox_data_bkt_m table for data with operation interval start equal to or before 2009-04-21 05:00:00     UTC
769 Apr 21 06:15:01 +0000 OX [     info]    - Processing the ox_data_bkt_r table for data with operation interval start equal to or before 2009-04-21 05:00:00     UTC
770 Apr 21 06:15:01 +0000 OX [     info]  Maintenance Distributed Engine Completed

I then checked for statistics and voila! I got stats!

Could someone confirm that the instructions here:
http://www.openx.org/en/docs/2.8/adminguid...uted+statistics

are not 100% correct?

[ChrisNutting]

You're right, the [lb] section tells the edge where to insert it's stats,

The SUPER priv should not be required (it should be used if available but should not cause maintenance to fail), it just improves performance on large systems by preventing the update bucket queries from being logged, as long as the bucket tables are being ignored by the slaves, the system should work fine.

I'll get the docs updated with the [lb] section and verify that the super permission shouldn't be required

[Mike Trest]

Chris or KKith or whomever:

I cannot find a GUI place to set the [lb] parameters. Can someone tell me where to find it in global config GUI.

I can, however, edit the conf.php values but really would like a template for guidance.

Can one of you (or someone) post an example as a templage for the [lb] section from a working config?
Meanwhile, I will poke and hack and perhaps I can stumble on something that works.

..mike..

[ChrisNutting]

You can't set it in the GUI, this setting could potentially be different on different delivery boxes (and does not need to be (and shouldn't be?) set on the UI box.

[Mike Trest]

You can't set it in the GUI, this setting could potentially be different on different delivery boxes (and does not need to be (and shouldn't be?) set on the UI box.

Thanks Chris regards UI box.

I am asking about delivery servers, each with local MySQL instance .
Here is template I am using (thanks to another OpenX user).
Are there any other related settings needed witn 2.8.0?

..mike..

[lb]
enabled=1
hasSuper=1
type=mysql
host=localhost
port=3306
username=xxxxxx
password="yyyyyyy"
name=ox
persistent=1
compactStats=1
compactStatsGrace=604800

[Flo]

I recently set up distributed statistics and had exactly the same problem. Took me quite some time to figure out which parts in the config file were missing. I finally copied them from one of the outdated documentation pages. It would probably ease the life of many new OpenX users if the 2.8 documentation would describe the process a bit better.

By the way, for my setup, the SUPER privilege was also required. (also took some time to figure out)

I just wish I had read this thread before installing. Would have saved a lot of time.