Introduction

Part one of this series covered the best Planning security query I’ve come up with to date. If you haven’t already checked it out, run the code, and then made whatever improvements to it you like as I encourage you to do so with this post.

This week’s installment isn’t going to be quite as groundbreaking (a bit of hyperbole on my part, how about just an extension of the hack) but it is both useful and the supporting code for next week’s post. What, you want me to give it all away at once? For that you are going to have to pay me. When it’s for free, it’s on my terms. Muahahahahaha. Hmm, the evil scientist laugh doesn’t really work all that well.

The problem

The query I put out last week is great for pulling all of the security in a Planning application but maybe not so great if you only want a portion of that security. And only want the security for a given group.

Ah, I hear you thinking, “Well, I’ll just run the query, import it into Excel, and filter what I want or don’t want.” Can I actually hear you thinking this? Why yes, as the writer of this blog, I have the power to hear the thoughts of my readers (all three or four of you, and “Hi, Mom!”) before they even think it. Insert evil scientist laugh. All self-delusion aside, that notion of filtering is a pretty reasonable thing to think. But you would be wrong.

Why oh why oh why would you be wrong?

The output of last week’s query is 76 lines long. Big deal, right? But that’s a cloud-based development environment. Real life size of that query’s result set is more like 20,000 lines. Ah, a bit different.

And then there’s another issue – if I want to know the direct and inherited security from all groups that end in EMEA, for instance, a filter in Excel simply isn’t going to provide that information. Here’s what I get when I do that filter:

Yes, I can see who is in the group PLN_CalcTest_Consol_EMEA, but what about any groups above it? Sorry Charlie, it isn’t going to get picked.

The reason is inheritance

No, not that private income you will receive when dear old Uncle Warbucks shuffles off this mortal coil, but instead Shared Services/Planning group inheritance. Cast your eyes down the page and you’ll see that users in the group PLN_CalcTest_Consol_EMEA also have whatever Planning rights are assigned to PLN_CalcTest_Consol and even PLN_CalcTest.

Big deal, you say, (Do you? Really? My powers of anticipation scare even me), show me that inheritance. Oh, but I will, and in fact I did last post, but let’s look at the Planning tables a wee bit more.

Users in groups query #1

Let’s first get a full list of users in groups.

SELECT

O1.OBJECT_NAMEAS "Group",

O2.OBJECT_NAMEAS "User"

FROM HSP_USERSINGROUP G

INNERJOIN HSP_OBJECT O1

ON G.GROUP_ID = O1.OBJECT_ID

INNERJOIN HSP_OBJECT O2

ON G.USER_ID= O2.OBJECT_ID

Users in groups query result #1

Group	User
PLN_CalcTest_Consol	TestPlanner1
PLN_CalcTest_Consol	TestPlanner2
PLN_CalcTest_Consol	JessicaC
PLN_CalcTest_Consol_Americas	TestPlanner1
PLN_CalcTest_Consol_APAC	TestPlanner2
PLN_CalcTest_Consol_EMEA	JessicaC
PLN_CalcTest	TestPlanner1
PLN_CalcTest	TestPlanner2
PLN_CalcTest	JessicaC

That query provides the group/subgroup relationship but what I am really interested in is just security for groups that end in “EMEA” and for that I need another, similar but different, query.

Users in groups query #2

SELECT

O1.OBJECT_NAMEAS "Group",

O2.OBJECT_NAMEAS "User"

FROM HSP_USERSINGROUP G

INNERJOIN HSP_OBJECT O1

ON G.GROUP_ID = O1.OBJECT_ID

INNERJOIN HSP_OBJECT O2

ON G.USER_ID= O2.OBJECT_ID

WHERE

O1.OBJECT_NAMELIKE'%EMEA'

Users in groups query result #2

Group	User
PLN_CalcTest_Consol_EMEA	JessicaC

Now I have just the group EMEA and the user (in the real world it would be multiple users) that is in that group. And with that I can combine the two queries and get, by user, all of the groups she is in.

Man and superman, or is that Group and supergroup?

Users in groups with CTEs

I hope you have now figured out that I am a huge fan of Common Table Expressions (CTEs) as a way of breaking up complex queries. But I don’t completely eschew subqueries as you can see in the WHERE and IN clause at the end of the UsersInGroups CTE clause.

-- Figure out the relationship is between users and groups

WITH

-- All users in groups

SuperSet AS

(

SELECT

O1.OBJECT_NAMEAS "Group",

O2.OBJECT_NAMEAS "User"

FROM HSP_USERSINGROUP G

INNERJOIN HSP_OBJECT O1

ON G.GROUP_ID = O1.OBJECT_ID

INNERJOIN HSP_OBJECT O2

ON G.USER_ID= O2.OBJECT_ID

-- Just the users in the specified group(s)

-- NB -- This could be expanded to multiple groups if need be

SpecificGroup AS

(

SELECT

O1.OBJECT_NAMEAS "Group",

O2.OBJECT_NAMEAS "User"

FROM HSP_USERSINGROUP G

INNERJOIN HSP_OBJECT O1

ON G.GROUP_ID = O1.OBJECT_ID

INNERJOIN HSP_OBJECT O2

ON G.USER_ID= O2.OBJECT_ID

WHERE

O1.OBJECT_NAMELIKE'%EMEA'

-- Use an IN statement with a subquery to limit the users from the

-- SuperSet CTE to the users in the SpecificGroup CTE

UsersInGroups AS

(

SELECT

S.*

FROM SuperSet S

WHERE S."User" IN(SELECT "User" FROM SpecificGroup)

)

SELECT

FROM UsersInGroups

Users in groups with CTEs result

Now, by user, I have a list of all of the groups that user belongs to by inheritance. Quite a bit different than the results of that Excel filter, isn’t it?

Group	User
PLN_CalcTest_Consol	JessicaC
PLN_CalcTest_Consol_EMEA	JessicaC
PLN_CalcTest	JessicaC

Take care of any individually assigned security

If you haven’t listened to, you may have assigned security directly to users. Sigh. I told you not to do that, didn’t I? Oh well, this wouldn’t be the first time in my life my advice has been ignored. Since I am the accommodating sort, this query will also grab all of the security that is individually assigned to users in the group that ends with “EMEA”. I’ll do that with the same subquery I used in the previous code snippet. I’m going to show you the CTE code, but no result set because I listened to my own advice and have only assigned security via groups.

UserDefinedSecurity as constrained by users in the CTE SpecificGroup

-- Get the security that is specifically assigned to users

-- Use an IN statement with a subquery to limit the users from the FinalCTE CTE to the users in the

-- SpecificGroup CTE

UserDefinedSecurity AS

(

SELECT

F."Type",

F."Object",

F."Security Type",

F."User/Group Name",

'User-assigned'AS "Parent Group",

F."Read/Write",

F."Hierarchy function"

FROM FinalCTE F

WHERE

"Security Type" ='User'

AND F."User/Group Name" IN(SELECT "USER" FROM SpecificGroup)

Putting it all together

Now that you’ve seen the code that gives the inherited security for a specific group as well as the users in those groups, it’s time to put the whole thing together and see just what, if anything, objects have security when the group ends in “EMEA”. Remember, this is all objects (even undocumented ones), explicitly assigned as well as inherited, by group and user.

Filtered Planning query

WITH

-- Dimensions and all of their members

Dimensions AS

(

SELECTDISTINCT

O.OBJECT_ID,

O.OBJECT_NAMEAS "Member",

(SELECT OB.OBJECT_NAME

FROM HSP_OBJECT OB

WHERE OB.OBJECT_ID= M.DIM_ID)AS "Dimension"

FROM HSP_OBJECT O

INNERJOIN

HSP_MEMBER M

ON M.MEMBER_ID = O.OBJECT_ID

--ORDER BY O.OBJECT_ID ASC

-- All of the other object types, including the ones that aren't documented or in

-- HSP_OBJECT_TYPE

ObjType AS

(

1 to 50 defined in HSP_OBJECT_TYPE

103 = Menus

107 = Composite forms

115 = Deployed Business Rule

116 = Looks like Business Rules, but don't exist in CM? So orphaned?

117 = Calculation Manager variables

118 = Business Rule Folder

119 = CalcMgrRulesets -- that's actually the OBJECT_NAME, so defined by system?

120 = There are four valies in a three Plan Type Planning app:

CalcMgrVariables

CalcMgrVariablesPTName1

CalcMgrVariablesPTName2

CalcMgrVariablesPTName3

SELECTDISTINCT

OBJECT_TYPE AS "OBJECT_TYPE",

TYPE_NAMEAS "TYPE_NAME"

FROM HSP_OBJECT_TYPE

UNION

SELECT

CONVERT(INT,'103')AS "OBJECT_TYPE",

'Menu'AS "TYPE_NAME"

UNION

SELECT

CONVERT(INT,'107')AS "OBJECT_TYPE",

'Composite'AS "TYPE_NAME"

UNION

SELECT

CONVERT(INT,'115')AS "OBJECT_TYPE",

'Business Rule'AS "TYPE_NAME"

UNION

SELECT

CONVERT(INT,'118')AS "OBJECT_TYPE",

'Business Rule Folder'AS "TYPE_NAME"

--ORDER BY OBJECT_TYPE, TYPE_NAME

-- Get every object in the application

ObjectID AS

(

SELECT

OBJECT_ID,

OBJECT_NAME,

OBJECT_TYPE,

SECCLASS_ID

FROM HSP_OBJECT

-- This is almost the end of the road, but it doesn't take into account implicit

-- security. Stop here if that isn't important

FinalCTE AS

(

SELECT

--OT.TYPE_NAME AS "Type",

-- If the OBJECT_TYPE = 50 then it is a user-defined or custom dimension

-- so do a subquery to pull the dimension name

CASE

WHEN O_ID.OBJECT_TYPE != 50 THEN OT.TYPE_NAME

ELSE (SELECT D."Dimension"

FROM Dimensions D

WHERE O_ID.OBJECT_ID= D.OBJECT_ID)

ENDAS "Type",

O_ID.OBJECT_NAMEAS "Object",

CASE

-- Subquery to get user or group type

(SELECT OA.OBJECT_TYPE

FROM HSP_OBJECT OA

WHERE OA.OBJECT_ID= AC.USER_ID)

WHEN 5 THEN'User'

WHEN 6 THEN'Group'

ENDAS "Security Type",

(SELECT OA.OBJECT_NAME

FROM HSP_OBJECT OA

WHERE OA.OBJECT_ID= AC.USER_ID)AS "User/Group Name",

CASE AC.ACCESS_MODE

WHEN 1 THEN'Read'

WHEN 2 THEN'Write'

WHEN 3 THEN'Write'

WHEN 4 THEN'Launch'

WHEN-1 THEN'Deny'

ENDAS "Read/Write",

CASE AC.FLAGS

WHEN 0 THEN'"'+ O_ID.OBJECT_NAME+'"'

WHEN 5 THEN'@CHI("'+ O_ID.OBJECT_NAME+'")'

WHEN 6 THEN'@ICHI("'+ O_ID.OBJECT_NAME+'")'

WHEN 8 THEN'@DES("'+ O_ID.OBJECT_NAME+'")'

WHEN 9 THEN'@IDES("'+ O_ID.OBJECT_NAME+'")'

ENDAS "Hierarchy function"

FROM ObjectID O_ID

INNERJOIN

ObjType OT ON OT.OBJECT_TYPE = O_ID.OBJECT_TYPE

INNERJOIN HSP_ACCESS_CONTROL AC

ON O_ID.OBJECT_ID= AC.OBJECT_ID

-- Figure out what the relationship is between users and groups

-- All users in all groups

SuperSet AS

(

SELECT

O1.OBJECT_NAMEAS "Group",

O2.OBJECT_NAMEAS "User"

FROM HSP_USERSINGROUP G

INNERJOIN HSP_OBJECT O1

ON G.GROUP_ID = O1.OBJECT_ID

INNERJOIN HSP_OBJECT O2

ON G.USER_ID= O2.OBJECT_ID

-- Just the users in the specified group(s)

-- NB -- This could be expanded to multiple groups if need be

SpecificGroup AS

(

SELECT

O1.OBJECT_NAMEAS "Group",

O2.OBJECT_NAMEAS "User"

FROM HSP_USERSINGROUP G

INNERJOIN HSP_OBJECT O1

ON G.GROUP_ID = O1.OBJECT_ID

INNERJOIN HSP_OBJECT O2

ON G.USER_ID= O2.OBJECT_ID

WHERE

O1.OBJECT_NAMELIKE'%EMEA'

-- Use an IN statement with a subquery to limit the users from the SuperSet CTE to the

-- users in the SpecificGroup CTE

UsersInGroups AS

(

SELECT

S.*

FROM SuperSet S

WHERE S."User" IN(SELECT "User" FROM SpecificGroup)

-- Get the security that is specifically assigned to users

-- Use an IN statement with a subquery to limit the users from the FinalCTE CTE to the

-- users in the SpecificGroup CTE

UserDefinedSecurity AS

(

SELECT

F."Type",

F."Object",

F."Security Type",

F."User/Group Name",

'User-assigned'AS "Parent Group",

F."Read/Write",

F."Hierarchy function"

FROM FinalCTE F

WHERE

"Security Type" ='User'

AND F."User/Group Name" IN(SELECT "USER" FROM SpecificGroup)

-- Get the security that is specifically assigned to groups

-- The join between the CTE UsersInGroups and FinalCTE is the key to implicit security

GroupDefinedSecurity AS

(

SELECT

F."Type",

F."Object",

F."Security Type",

U."User" AS "User/Group Name",

F."User/Group Name" AS "Parent Group",

F."Read/Write",

F."Hierarchy function"

FROM FinalCTE F

INNERJOIN UsersInGroups U

ON U."Group" = F."User/Group Name"

-- UNION the explicit to the user and the implicit via a group security

UserAndGroupDefinedSecurity AS

(

SELECT

FROM UserDefinedSecurity

UNION

SELECT*FROM GroupDefinedSecurity

)

-- Now report out however you like

SELECT

"User/Group Name",

"Security Type",

"Parent Group",

"Type",

"Object",

"Read/Write",

"Hierarchy function"

FROM UserAndGroupDefinedSecurity

ORDERBY 1, 4, 3, 5

The full result set

That’s 26 rows in case you haven’t been counting. Last week’s post returned 75 rows.

Where do we go from here?

Let’s review:

The last post covered a query of all Planning objects, documented and otherwise.
This post is a refinement of the above but just for specific users and groups, and quite necessary for the next post, which will be all about…
This post’s query but in Planning’s ImportSecurity.cmd’s SECFILE.txt format. I’ll also tell you why you should use this in the real world to migrate security instead of LCM.
And finally, and maybe most excitingly (I use the word “excitingly” in a sort of liberal matter), we’ll take all of this and write out MaxL code to assign filters.

Who knew that SQL could be so useful? If you were at Kscope13 and attended my Practical SQL for EPM Practitioners session you’d know all about this, and more. Pity you weren’t there but if you did attend the conference, you can download the PowerPoint-made-into-pdf here.

Speaking of Kscope13 sessions

Jessica Cordova and I copresented a session titled Top Six Advanced Planning Tips. There’s a wee problem with that title, however – we only presented three sessions. The goal wasn’t to cheat anyone; we simply had way too much good content and couldn’t fit it into 50-odd minutes. We’re committed to giving the ODTUG community all that we know and we shall do so through a two part ODTUG webinar.

If you’re interested in hearing all six of the tips, you can do so from the convenience of your office, first on 6 August 2013 at 12 noon Eastern, and then on 13 August again at 12 noon Eastern. Here’s what we’re going to cover:

Planning for Success
Using an Inherited Security Model (here I will do my finger wagging in person)
Bringing Metaread Filters to Planning
Melding and Managing Forms, Task Lists, and Process Management
Making Planning Aggregations Better With Focused Aggregations
Automated LCM Migrations

Be seeing you.

Introduction

Two exciting (okay, I may be stretching the meaning of the word “exciting”) bits of Planning news this week.

Updated Planning security queries

I have expanded and modified part two and three of my Planning security query series. Would you believe that I conceived of the modifications whilst shoveling gravel to fill in potholes? No, this was not part of my prison-release program (note to everyone – that’s a joke, I have never been arrested, let alone convicted of anything, except being an Essbase/Dodeca/ODI/SQL geek and I am judge, jury, and executioner on that sentence), but instead what my bored mind was doing as I was trying to see if my shoulder really would separate if I picked up enough gravel with a shovel. Note – so far, no replacement shoulder, so that gym time has paid off.

In any case, as I was putting in my geeks-who-do-yardwork-for-other-people time, I had nothing much to actually think about (some call it thinking) and I started naturally (?) thinking about what the queries I had written for this series. And then I realized – hey, I can improve these quite a bit by changing some of the CTEs and making the queries that much better. And so I have. Go back and take a look – I hope you’ll agree that the output is now more useful.

Changed time for Too Big for Kscope13 – Part 2 of Six Advanced Planning Topics

Continuing the theme of me not thinking (something I appear to be quite good at), this webinar was originally scheduled for tomorrow, 13 August 2013 at 12 noon Eastern – it is now scheduled for Wednesday, 14 August 2013 at 12 noon Eastern. In other words, it is being held a day later. And that’s my fault. Oops.

I am travelling to my client (yes, I work and for some reason clients pay me for it) on Tuesday. I completely forgot about this altered travel pattern (my usual on site schedule is Monday through Thursday but this was a client-requested exception) when I agreed to the webinar date. However, I did oh-so-cleverly manage to remember this in the middle of a haircut last week. What do you think about during a haircut? Not much, right? Well, in case you ever have the same brainwave, do not jump up in the middle of a haircut with an, “Oh, crap!” exclamation or you will receive the same, “WTH is your problem, Cameron?” response. Oops again.

Hopefully you were there for Part 1 where Jessica and I covered general Planning implementation good practices, inherited security, and bringing metaread filters to Essbase datasources in Planining.

Part 2 should be just as interesting as it will cover:

Melding and Managing Forms, Task Lists, and Process Management
Making Planning Aggregations Better With Focused Aggregations
Automated LCM Migrations

I hope you can all make it to the slightly changed time and sorry to all for any inconvenience.

What’s next and a question

There’s one more Planning security query on offer – the creation of ASO (well, it could be for BSO but why?) reporting cube filters. Hopefully that will be out in the next few weeks.

And I have a question for you, gentle reader, about a potential Kscope14 session. Would you be interested in a session that goes under the covers of EPM and pulls out all kinds of information? By that I mean I would include Planning but I’d also go after Shared Services, BI+ (Financial Reports), Calculation Manager, and EPMA. The session would be a series of use cases, code examples (with the bits that were a total PIT-you-know-what highlighted), and sample output. Is this something the ODTUG world would be interested in? Let me know care of this blog.

Where oh where oh where will you be at Oracle OpenWorld 2013?

Presumably you’ll be running around like a chicken with your head cut off. That’s the point behind conferences, right? So maybe nowhere, then. And How Can You Be In Two Places At Once When You’re Not Anywhere At All? Hmm, that sounds strangely existential and I’m pretty sure OOW 2013 isn’t the place to gaze at your navel and contemplate the universe. It’s a whirlwind of meetings, sessions, seminars, councils, and just a busy, busy, busy time. Is there any opportunity to relax in what has got to be one of the largest and hectic conferences extant?

Tim and Cameron’s Excellent Essbase/EPM Oracle OpenWorld 2013 meetup

Why yes there is. Last year I wrote about our inaugural meetup here in this very blog. It was a smashing success (no, no plates were broken in the course of the event although I was as always ready to let out my inner Zorba the Greek– or is that Geek?) in that it was informal, relaxed, and just about perfect meetup with a bunch of mostly Essbase (although all colors of the EPM spectrum are welcome) geeks and Oracle’s EPM management.

Yup, you read that right. I personally sat across from the head of EPM product management and had the Essbase development manager to my right. I don’t know about you but I don’t typically get to talk to these guys in anything other than a formal setting. Come to think of it, I don’t get to talk to them in a formal setting either. So it was sort of special event. And it could be for you as well.

We’re going to try to recapture lightning in that proverbial bottle and do it again. Of course I cannot guarantee that there will be anyone at this event other Tim Tow and myself as we are certainly not the masters of the EPM universe. Hopefully some people from the Oracle EPM world can make it – I think they enjoyed it as much as we geeks on the outside of Big Red did.

What, how, when, and where?

What

To get a feel of how many people are coming to the event, I have created a meetup event: The best Essbase meetup at OOW 2013. 24 September, 7pm, Specchio-Ristorante. In true geek fashion I endeavored to cram as much information as possible into the title but I’ll break it out for everyone not into decoding fields.

Also, I should note that this is a pay-your-own-way event. It’s not an official Oracle anything so if you have an invite to a consulting company dinner and you like to live high on the hog, the Essbase/EPM meetup might not be for you. But I’ll bet our conversation is way more interesting. :)

How

The meetup link is how you répondez s'il vous plait to the event so we have an idea of how big this particular breadbox is. Please do sign up if you intend on coming so we can plan.

We are trying to limit this to about 50 to 75 people as the physical space, while spacious, is not the size of HMS King George V.

When

7 pm, Tuesday, 24 September 2013.

Where

It’s at Speccio-Ristorante, 2331 Mission St. (between 19th St & 20th St), San Francisco. They have excellent Venetian food, coffee (did I mention I like coffee?), and of course a bar.

Join us, won’t you?

I think I’ve given you every bit of information I can and enthused about this as breathlessly as I can. In other words, I have shot my meetup promotional bolt. Hopefully it found its target and you’ll come to the the event.

Be seeing you.

Introduction

Why, yes, he can, much to his (and maybe everyone else’s) amazement.

Those of you who have followed this blog (ah, Mum, you do still follow the blog, right?) know that I am, uh, infrastructurally challenged. Yes, I am able to, somehow, make a living at this stuff implementing solutions based on Essbase, Planning, ODI, and various and sundry SQL hacks. But installing this stuff? Alas, not so much. (Alas because it is quite remunerative and even more than that, when I get to a client that has the EPM Environment From Hell, I could help make it better.)

So it is with not a small amount of trepidation that I approached the 11.1.2.3 install this week. Yes, yes, I know, it came out in April, and world+dog already has it installed, but I’ve been on projects, went to New Zealand/Australia to help run ODTUG’s SP conferences, had this wee event known as Kscope13 come up (remember that I am on the ODTUG BoD, am responsible for the technical side of the labs – also note that I did not do the Amazon installs, had a total of six presentations – some before Kscope, others were private, participated on two panels, I could go on but you get the idea), and have in general been running around like a one armed paperhanger.

But all of that, thankfully, has passed, and I am now able to put my mind to an install. My last go round with 11.1.2.0 was on my old Dell Latitude e6400 (I bought the maxxed out 64 bit, 8 GB RAM model) was not exactly a success. I’ve turned to the Amazon Cloud in the meantime and it has been great. But all along I’ve wondered – if I had a Big Enough Box, could I have gotten away with doing this all in a VM? During my Latitude days, I worked with Dan Pressman to prove that we couldconvert John Booth’s 11.1.2.2 AMI to VMWare (and someone converted that to Oracle’s Virtual Box) but of course 8 GB of RAM is simply not adequate.

However, I splurged earlier this year on a Dell Precision M4700 laptop (lest you get jealous, remember that while I have the privilege of picking whatever gear I want, I also have the responsibility of paying for it – ah, the carefree life of an independent consultant) and it has 32 GB of RAM and a superfast (if a bit small) SSD drive. What would happen if I installed everything from scratch? Could I handle it? Good question.

What took the longest

Believe it or not, the bit of the install that took longer than anything else was the install and patching of Windows 2008 R2 Enterprise, SQL Server 2008 R2, and Office 2010 (32 bit). Yes, these are all legitimate licenses (remember that bit above about making decisions and then paying for them) that I get through Microsoft’s Action Pak program for MS Partners. DVD ROMs simply aren’t all that fast (I never did go for those .iso images although I have to wonder given how long it took to install).

Downloading, now that I no longer have the slowest “broadband” connection in the world (DSL was fast in 1999, not so much when the world changes round you), EPM 11.1.2.3 was pretty fast. I was able to pull down much more than I needed for my EPM install (I downloaded just about everything off edelivery in 11.1.2.3) in the space of an afternoon while the MS world was installing. So far, so good.

The actual install

I’m not going to cover how to do this because you can:

Read the documentation (gasp, and I didn’t, sob, do it, partially on purpose and partially because I am lazy). It bit me in the you-know-what a bit later as you shall see if you but read on.
Read John Goodwin’s blog on 11.1.2.3 installation. Also see his post on separately installing the ODI Console and Studio– I have not yet done this but it is on my list of things to do. Lastly, take a look at his post on Essbase PSUs– that too is on my list of Things To Do.
Read the Finnish Hyperion Guy’s post on Rapid Deployment. Although in the end I did a full install as I wanted bits outside of either the Planning or Essbase rapid deployment it’s still a good read. Also, every time I see his web site come up I am reminded of Sibelius’Finlandia. Suomi!

A slight digression

Okay, true story that is tangentially related to this post: When I was living in Europe, one of my buddies (I’ve known him since 8th grade) came over for a visit to the land of Cheese and Beer and then continued on to Finland. He was (and is) a huge Sibelius fan and knowing not one single word of Finnish when he got out of passport control he went to the taxi rank and said to the driver two words: “Sibelius, Finlandia”. The driver took him here. How cool is it that a classical composer is such a national hero that everyone knows his name. And that one of his best known pieces (Finlandia) is an act of rebellion against an imperialist power? Try that in the US of A with “Copland, Rodeo” even if most of us (I am making a bit of a leap here) would at least recognize the music.

Clawing my way back to relevancy

So what to say about the install? I used SQL Server instead of Oracle (sorry, but it just easier for me although I have a feeling that I will regret it when I get round to installing all the components of ODI – I know when I created an Amazon AMI with ODI and SQL Server Express it was a total stinker) so I differ somewhat from John G’ and Henri’s path.

The other thing I didn’t do was read the documentation. Do that, won’t you? Here’s the bit that I missed on MS SQL Server (see, I told you I was going to be sorry I didn’t use Oracle) database creation requirements.

If you, like me, don’t end up doing this, the following SQL queries will:

Tell you how stupid you were for not reading the docs
Fix it the way you should have before you did the install

Stupid, stupid, stupid

This is bad:

You want those to be 1’s, not 0’s.

The fix

I reached out to John Booth (John, believe it or not, I really constrain myself from asking you question as I would otherwise have to set up a retainer account with you. And I hear you are a bit busy. Thanks again.) on this one and he pointed out that this was noted on a Network54 thread.

Submit this query and all should (hopefully) be well:

Fwiw, I had issues with getting the snapshot isolation to take – it ran and ran and ran and never seemed to finish. I killed it and then went looking and found this thread on StackOverflow re figuring out if this would ever finish or not. I ended up stopping all of the EPM services and reissuing the command and all was good.

I should note that while the formal documentation states to set this for all EPM SQL Server databases I found that I only needed to do it for EPMA.

Here’s what it all looks like when things are working:

And that’s it

It’s running. And quite nicely, too.

I don’t know if the above is blind luck (not all that likely), adequate resources (quite a bit more likely), or a reflection on Oracle’s part to make at least compact deployments easier to do (winner, winner, chicken dinner). Thanks, Oracle, for making this a wee bit simpler. I just want to use the tools, not be an infrastructure expert. Infrastructure just isn’t my particular cup of tea and now I don't need to drink it.

What’s next?

I have to get all of ODI installed and apply the patches I noted. After that…start working on those Kscope14 sessions. All on my laptop. Forgive me Amazon, because I will not be dumping the Cloud but honestly, AWS can be a bit expensive for a one man band.

Be seeing you.

Just how do ASO Planning reporting cubes work?

Well, I could retype (in my own words, ‘natch, as I am no plagiarizer) the Planning administration guide but instead I will direct you to The Great and Good John Goodwin’s as-always excellent explanation right here.

Did you read it? All pumped up? Ready to deploy an ASO reporting application? Did you catch the bit that John missed? Actually, that’s not fair, John didn’t miss anything. Did you catch what Oracle missed?

The missing link

What’s missing is the security side of reporting. Unless of course you are planning (oh, I slay me with my puns, but maybe no one else) on giving access to everyone, or hand crafting filters. Neither of which are much use, or fun.

What do I mean by missing security? Well, you have built your reporting cube, loaded it with data, scripted the whole thing but where are the filters for that supercalifragilisticexpialidocious reporting database? The answer is: nowhere because they don’t get created. Oops.

And of course you have created all kinds of rules based on usernames and groups within Planning (you have been paying attention during thelastthree posts on Planning security, right?) Which is all now a bit pointless as those dimensional security rules only apply to Planning because the definitions are in, wait for it, Planning.

Whew, a bit tautological, but you get the idea – Planning security is like the Hotel California. Or maybe the Roach Motel. Once in, it ain’t getting out. Except of course with the power of SQL and the queries I have given you, in fact you can get security out of Planning and apply it elsewhere. How? Read on, gentle reader.

A solution in two parts

Part the first

For those of you smart (or foolish or merely desperately sad) who follow my blog posts on Planning security know that with a query that figures out what rights a Planner has using both explicit and implicit, group-defined security, you can get the bones of an Essbase reporting application’s filter definition.

Think about it – you can write a query that knows everything, and I mean everything, about dimensional (and other, but only dimensions are germane to this issue) security from a Planner perspective. If the output from that query looks like a MaxL filter security assignment, you have the guts of a fully dynamic, programmatically derived Essbase filter.

Part the second

Once you derive the filter you have to assign it to the Essbase application/database. I’m not going to show you how to do that in this blog because:

Running this MaxL from a msh nesting command keyword in your own MaxL script is really easy to do
If you don’t know how to do the above maybe you shouldn’t be trying this
The solution that I implemented is pretty specific to the client.

High level overview of the scripting

To address the bit that is specific to the client, without of course naming the (un)lucky recipient, realize that I had no access to proper scripting and had to improvise. Dan Pressman gave me the suggestion to use SQL*Plus as a scripting environment to generate the script. It worked a treat – and is completely useless if you are doing this on SQL Server. Roll your own or contact me care of this blog and we can discuss the gory details off line.

The query

A caveat

One thing to note – when I ran this in PL/SQL, I used the undocumented-but-out-there command wmconcat to give me a comma delimited filter. You should probably use LISTAGG but the need to go over the varchar limit of 4,000 bytes and my inability at the client to write a custom function to point LISTAGG to a CLOB data type mean that wmconcat was my only option.

Interestingly (well, interesting to me), a CLOB can contain up to two gigabytes of information. I was limited by the 32K size limit of SQL*Plus for a column.

As far as I can tell, in SQL Server the sort-of equivalent of wmconcat/LISTAGG is FOR XML PATH and its limit seems to be two gigabytes as well.

Ah, the joys of an almost-standard.

The code

WITH

Dimensions AS

(

SELECTDISTINCT

O.OBJECT_ID,

O.OBJECT_NAMEAS "Member",

(SELECT OB.OBJECT_NAME

FROM HSP_OBJECT OB

WHERE OB.OBJECT_ID= M.DIM_ID)AS "Dimension"

FROM HSP_OBJECT O

INNERJOIN

HSP_MEMBER M ON M.MEMBER_ID = O.OBJECT_ID

ObjType AS

(

2 = Dimension

31 = Scenario

32 = Account

33 = Entity

34 = Time Period

35 = Version

37 = Currency

38 = Year

50 = User Defined Dimension

SELECTDISTINCT

OBJECT_TYPE AS "OBJECT_TYPE",

TYPE_NAMEAS "TYPE_NAME"

FROM HSP_OBJECT_TYPE

WHERE OBJECT_TYPE IN('2','31','32','33','34','35','37','38','50')

ObjectID AS

(

SELECT

OBJECT_ID,

OBJECT_NAME,

OBJECT_TYPE,

SECCLASS_ID

FROM HSP_OBJECT

FinalCTE AS (

SELECT--DISTINCT

--OT.TYPE_NAME AS "Type",

CASE

WHEN O_ID.OBJECT_TYPE != 50 THEN OT.TYPE_NAME

ELSE (SELECT D."Dimension"

FROM Dimensions D

WHERE O_ID.OBJECT_ID= D.OBJECT_ID)

ENDAS "Type",

O_ID.OBJECT_NAMEAS "Object",

CASE

-- Subquery to get user or group type

(SELECT OA.OBJECT_TYPE

FROM HSP_OBJECT OA

WHERE OA.OBJECT_ID= AC.USER_ID)

WHEN 5 THEN'User'

WHEN 6 THEN'Group'

ENDAS "Security Type",

(SELECT OA.OBJECT_NAME

FROM HSP_OBJECT OA

WHERE OA.OBJECT_ID= AC.USER_ID)AS "User/Group Name",

CASE AC.ACCESS_MODE

WHEN 1 THEN'Read'

WHEN 2 THEN'Write'

WHEN 3 THEN'Write'

WHEN-1 THEN'Deny'

ENDAS "Read/Write",

CASE AC.FLAGS

WHEN 0 THEN'"'+ O_ID.OBJECT_NAME+'"'

WHEN 5 THEN'@CHI("'+ O_ID.OBJECT_NAME+'")'

WHEN 6 THEN'@ICHI("'+ O_ID.OBJECT_NAME+'")'

WHEN 8 THEN'@DES("'+ O_ID.OBJECT_NAME+'")'

WHEN 9 THEN'@IDES("'+ O_ID.OBJECT_NAME+'")'

ENDAS "Hierarchy function"

FROM ObjectID O_ID

INNERJOIN

ObjType OT ON OT.OBJECT_TYPE = O_ID.OBJECT_TYPE

INNERJOIN HSP_ACCESS_CONTROL AC

ON O_ID.OBJECT_ID= AC.OBJECT_ID

-- Commented out, but this is where you might limit the scope of the dimensions by name

EmployeeEntity AS

(

SELECT

FROM FinalCTE

--WHERE

-- ("Type" = 'Employee'

-- OR "Type" = 'Entity'

-- OR "Type" = 'Geography')

-- --AND "Security Type" = 'Group'

UsersInGroups AS

(

SELECT

O1.OBJECT_NAMEAS "Group",

O2.OBJECT_NAMEAS "User"

FROM HSP_USERSINGROUP G

INNERJOIN HSP_OBJECT O1

ON G.GROUP_ID = O1.OBJECT_ID

INNERJOIN HSP_OBJECT O2

ON G.USER_ID= O2.OBJECT_ID

UserDefinedSecurity AS

(

SELECT

FROM EmployeeEntity

WHERE "Security Type" ='User'

GroupDefinedSecurity AS

(

SELECT

E."Type",

E."Object",

E."Security Type",

U."User" AS "User/Group Name",

E."Read/Write",

E."Hierarchy function"

FROM EmployeeEntity E

INNERJOIN UsersInGroups U

ON U."Group" = E."User/Group Name"

UserAndGroupDefinedSecurity AS

(

SELECT

FROM UserDefinedSecurity

UNION

SELECT*FROM GroupDefinedSecurity

)

SELECTDISTINCT

'create or replace filter "appname"."dbname"."f'+ U1.[User/Group Name] +'" meta_read on '+''''+

STUFF(( SELECT','+ U2."Hierarchy function"

FROM UserAndGroupDefinedSecurity U2

WHERE U1.[User/Group Name] = U2.[User/Group Name]

FORXMLPATH(''),TYPE).value('.','VARCHAR(max)'), 1, 1,'')

+''''+' ; '

+'grant filter "appname"."dbname"."f'+ U1.[User/Group Name] +'" to '+ U1.[User/Group Name] +' ; '

AS'DefineAndGrantFilter'

FROM UserAndGroupDefinedSecurity U1

/* Oracle PL/SQL code

SELECT

('create or replace filter "appname"."dbname"."f' || "User/Group Name" || '" meta_read on ' || ''''

|| wm_concat("Hierarchy function")

|| '''' || ' ; ') ||

('grant filter "appname"."dbname"."f' || "User/Group Name" || '" to ' || "User/Group Name" || ' ;') AS "DefineAndGrantFilter"

FROM UserAndGroupDefinedSecurity

GROUP BY "User/Group Name"

The MaxL code

Write (this is the bit I am not including as it was so specific to a very narrowly defined environment) this code out to a text file, then pull it in via MaxL’s msh nesting command:

You are actually seeing two lines of MaxL code for every user: one to create or replace the filter, the other to grant it to a specific user.

And yes, that is the inherited and explicitly set security for all four users in my little test application. Pretty cool hack, is it not?

So why isn’t this part of Planning?

Up to Planning 11.1.2.3, I think the answer to that question is because Oracle have no idea what you are doing with your reporting application. The above query generates MaxL to write filters and grant them which makes perfect sense so long as your reporting application mimics the dimensionality of the BSO Planning Plan Types.

But what happens when your reporting application goes beyond what Planning has? How could Oracle have tied the filters to an unknown? The answer of course is they could not, and did not, and will not going forward. Knowing the unknowable is sort of difficult.

11.1.2.3 is a horse of a different color – with ASO Plan Types one could argue that the ASO database now generated by Planning is good for both input and reporting (I am leaving aside the fact that a lot of BSO Planning Plan Types have been used for just that since The Year Dot). And if your ASO Planning cube is generated by Classic Planning (the only choice, btw), then it follows that the filters your Planners need will be there. So these filters are there finally.

But for everyone else not on 11.1.2.3, the above technique makes sense and who knows – you may want to take that Planning security definition and apply it to other databases. You now have a query that will do just that.

I hope you enjoyed the hack.

Be seeing you.

What’s it all about, Essbase Hacker?

I have this nifty cool Oracle OpenWorld (henceforth referred to as “OOW”) Press/Blogger badge. This means that, somehow, and very possibly against Oracle’s better judgment, it is in fact true that someone beyond my mother reads my blog. Or at least believes my claim that I in fact have a blog. Well, I do, and the fact that you are reading this is proof positive.

Photo credit: Natalie Delemar

To honor Oracle’s possibly poor decision making skills, I am going to endeavor to live blog OOW. I’ve tried doing this at Kscope but have run up against the fact that I am really, really, really busy at that conference and there simply isn’t any time to do the blogging. Happily, I am responsible for much less (just a shared presentation with my MMIC*/older-brother-I-don’t-actually-have-and-who-denies-any-familial-relationship Glenn Schwartzberg) and so I actually get to attend OOW. I can hardly wait.

The sessions

OOW is quite serious about signing up for sessions. If you have not signed up for a session, and there is room, you can attend. But if the session is filled, and you again didn’t sign up…well, there is always a waiting list and you get to hope that someone changes his mind. And yes, for future OOW 2014 attendees, you need to sign up in advance for the more popular sessions or you will end up on that waiting list. Like me.

Here’s what I have planned (I hope) for this year thus far:

Subject	Start Date	Start Time	End Date	End Time
UGF9945 -- Welcome to Oracle Hyperion 11.1.2.3, Oracle Business Intelligence Enterprise Edition 11.1.1.7	9/22/2013	8:00	9/22/2013	9:00
UGF9927 -- What’s New in Oracle Hyperion Financial Management and Close Solutions in Release 11.1.2.3	9/22/2013	9:15	9/22/2013	10:15
UGF9929 -- Oracle Essbase 11.1.2.3, Oracle Business Intelligence Enterprise Edition 11.1.1.7: What’s New	9/22/2013	10:30	9/22/2013	11:30
UGF9946 -- What’s New in Oracle Hyperion Planning and Budgeting Solutions 11.1.2.3	9/22/2013	11:45	9/22/2013	12:45
UGF9721 -- Agile Real-Time BI with Oracle Business Intelligence/Oracle Data Integrator/Oracle GoldenGate	9/22/2013	13:00	9/22/2013	14:00
UGF9947 -- Oracle Hyperion Smart View for Office, Oracle Hyperion Financial Reporting 11.1.2.3: What’s New	9/22/2013	14:15	9/22/2013	15:15
UGF9911 -- Build a Performance Layer for Oracle Business Intelligence Applications 11g	9/22/2013	15:30	9/22/2013	16:30
KEY11026 -- Welcome Keynote	9/22/2013	17:00	9/22/2013	19:00
KEY11045 -- Transforming Businesses with Big Data and Analytics	9/23/2013	8:00	9/23/2013	9:45
CON9328 -- Using Oracle Business Intelligence with Oracle Essbase: New and Upcoming Capabilities	9/23/2013	10:45	9/23/2013	11:45
GEN9523 -- General Session: Empowering Modern Enterprise Performance Management	9/23/2013	12:15	9/23/2013	13:15
CON10943 -- Mobile and Web-Based Reporting for Oracle Hyperion	9/23/2013	13:45	9/23/2013	14:45
CON5122 -- Oracle Business Intelligence Enterprise Edition Integration with Oracle Hyperion Datasources	9/23/2013	15:15	9/23/2013	16:15
CON9178 -- Oracle Essbase New Features and Roadmap Update (waitlisted)	9/23/2013	16:45	9/23/2013	17:45
KEY11046 -- Hardware and Software, Engineered to Work Together	9/24/2013	8:00	9/24/2013	10:00
CON9498 -- Tips and Tricks: Oracle Hyperion Enterprise Planning Suite	9/24/2013	10:30	9/24/2013	11:30
CON9500 -- What’s New and What’s Coming: Oracle Hyperion Enterprise Planning Suite	9/24/2013	12:00	9/24/2013	13:00
KEY11049 -- Cloud Keynote	9/24/2013	13:30	9/24/2013	15:15
CON9277 -- Oracle Business Intelligence Product and Technology Roadmap	9/24/2013	15:45	9/24/2013	16:45
CON9516 -- Customer Success: EPM on Oracle Exalytics	9/24/2013	17:15	9/24/2013	18:15
KEY11047 -- Modern Customer Experience: Welcome to the Age of the Customer	9/25/2013	8:00	9/25/2013	9:45
CON9513 -- Oracle Enterprise Performance Management in the Cloud	9/25/2013	10:15	9/25/2013	11:15
CON8993 -- Master Data Management Best Practices, Data Quality, Data Governance, and ROI	9/25/2013	11:45	9/25/2013	12:45
CON8740 -- Real-World Performance: What Is Big Data, and What Makes It Big?	9/25/2013	13:15	9/25/2013	14:15
CON9521 -- Product Development Panel Q&A: Oracle Hyperion EPM Applications	9/25/2013	15:30	9/25/2013	16:30
CON9514 -- Oracle Enterprise Performance Management on Mobile	9/25/2013	17:00	9/25/2013	18:00
CON9287 -- Mining Social Data for Insights	9/26/2013	9:00	9/26/2013	10:00
CON9352 -- Oracle Exalytics Technical Architecture and Deep Dive	9/26/2013	11:00	9/26/2013	12:00
CON9420 -- Configuring Oracle Business Intelligence Applications on Oracle Data Integrator: A Deep Dive	9/26/2013	12:30	9/26/2013	13:30
CON9493 -- Taking Oracle Essbase Beyond Finance	9/26/2013	14:00	9/26/2013	15:00
CON6456 -- Maximize Oracle Hyperion Planning Performance with Oracle Exalytics	9/26/2013	15:30	9/26/2013	16:30

My stalkers can now follow me to their heart’s content.

The blow by blow account

Sundays at Oracle OpenWorld (henceforth referred as “OOW”) are where user groups get to shine. As an ODTUG bigot/evangelist/fan club member, of course I am at going to be at ODTUG’s Sunday Symposiums.

The man formerly known as my employer

I am in the first session, listening to Edward Roske, CEO of interRel Consulting, LLC, talk about all the new and cool things in 11.1.2.3. As always, Edward is a droll and amusing presenter and really is providing a good overview of the 11.1.2.3 stack which seems only to increase in size year over year.

From a career perspective, I am only a little concerned that I don’t do, oh, the majority of these products but a man has to figure out what he can do, and do well.

Financial Close

Remember that comment above about me not being an active (or for that matter, inactive) practitioner? Sadly, this session is all about just that.

Alexandre Seran of ADI Strategies is presenting the latest and greatest re the 11.1.2.3 Financial Close suite (which is way more than HFM). Should I attempt to get into this stuff? Can I remember anything from my five accounting classes from oh so long ago? C’mon blog readers, what do you think? Cameron’s blog for HFM hackers – does that have a nice ring to it?

Essbase/OBIEE What’s New

Glenn Schwartzberg (MMIC, older brother but not really, etc.) is actually talking about something I do have half, or maybe half of a half, of a clue about, a tool that is near and dear to my heart, and is the primary focus of this blog: Essbase.

Older not-bro is talking about all of the cool stuff in 11.1.2.3. He claims that users will never see the changes but we Essbase hackers will, most definitely. I for one have got to write a blog post (in case you have ever wondered about why I write this stuff, in part is because it forces me to really figure out and why things work) about the overall performance of the different ways of block creation -- @CREATEBLOCK is cool stuff.

OBIEE is now up for review. Glenn says that Essbase and OBIEE integration is “astounding”. Hopefully not in a bad way. OBIEE is yet another one of those tools I don’t “do”. Should this blog be named Cameron’s Blog for OBIEE Hackers? Inquiring minds want to know.

What’s new with Hyperion Planning

Deanna Sunde, the ODTUG Hyperion SIG President started in 11.1.2.1’s triumvirate of Essbase/Planning/Smart View, moved on to 11.1.2.2, and has ended up in 11.1.2.3 Planning.

For someone who has seen Planning since version 1.5 I sometimes forget how much the tool has evolved. How much of the latest and greatest functionality to I use? I would guess about 85 percent. It isn’t a question of not wanting (or wanting) to use these features but if I can get them to be approved and used by a customer. I for one am glad that the tool continues to get small (decimal precision setting) and big (ASO Planning) enhancements.

What’s new with Hyperion Financial Reports and Hyperion Smart View

Glenn Schwartzberg (Are you getting the idea that there is a bit of hero worship going on here? Or maybe just that Glenn is a consummate professional who really, really, really knows the BI/EPM products? ) graciously (and his employer, interRel, was as well as they produced the original content that I only slightly modified) invited me to copresent in this session.

Here we are – Glenn’s the good looking one.

What does that all mean? I did the Financial Reports side of it, Glenn did Smart View. And from an FR perspective, that meant mobile. Believe it or not, I once worked for a Fortune 50 firm that produced fancy looking briefing books from its mainframe-based Executive Information System (EIS). These briefing books went to the Company Group Chairmen and Sector Vice Presidents who then managed the business on a global scale. When one of those executives wanted a new version of the report (the unbelievably fancy for 1990 reports were printed, in color, weekly, even though the base data was updated daily; distribution by paper was so onerous and expensive only special requests went outside of that weekly report refresh), he (or more likely his assistant) called my boss, who told me to run the report, print it out in the corporate print shop, and then physically walk (quickly) to the executive in charge. Whew. In other words, I was the mobile solution. It wasn’t exactly what I went to university for but I did realize I was at the bottom of the totem pole.

And now that’s all gone and done, and for once this sort-of traditionalist is glad to see it go. The mobile access is so cool I may even go out and buy an iPad, much to the joy of my family that doesn’t understand why we have to use tube radios, write with goose feather quills, and eschew electricity (except for my laptop).

Oracle ACE Dinner

Every year at OpenWorld, OTN gives all of the ACEs and ACE Directors a chance to get together on the Sunday of the conference. This year was at the Walt Disney Family Museum which, if you are a fan of classic animation (and I am) a fascinating view of the private life of Walt Disney.

I saw my first honest-to-goodness Oscar her – in fact many of them. Here is an extraordinarily cool one that Disney won for Snow White and the Seven Dwarfs:

Quite the inspired choice of venues and the food was good as well.

What’s next?

You’ll hopefully see me update this during the day and the whole post will be capped by the Oracle ACE dinner tonight. With pictures, even.

Actually there is no next, except me going to sleep. OMG, this time zone difference is making me sleepy. Time to hit the sack.

Be seeing you.

*MMIC = My Man In California. Glenn is my dogsbody/majordomo and does all of my EPM heavy mental lifting. Although he thinks the relationship works the other way round.

Now we get to the meat

If Sunday was user group time, today marks the real start of OpenWorld. By that I mean that yesterday’s presentations were approved by the various user groups, including ODTUG and today’s sessions are approved by Oracle.

Using Oracle Business Intelligence with Oracle Essbase: New and Upcoming Capabilities – Moscone South - 302

Ramke Ramakrishnan - Principal - Business Advisory Services, MarketSphere Consulting, LLC; Gabby

Rubin - Sr. Director, Product Management, Oracle;

10:45 am to 11:45 am

This is a mix of Oracle current state (and a bit of the new stuff). I was particularly impressed with the Mobile BI model. I may actually, finally, and most reluctantly (not because I am a Luddite, but because I am cheap) going to be forced to buy an iPad. That’s two reasons (yesterday’s mobile FR) and today BI.

I am quite excited about what I see as the replacement for Web Analysis – this is the Application Designer tool which bypasses most of the formal stuff in OBIEE and instead gives a drag and drop UI against Essbase. My buddy Natalie Delemar says Oracle have been describing this for a long time but I for one finally get it. I suppose I had to see it. And maybe that’s the literal truth – I am 99% sure this was demoed at Kscope13 but I was just too busy (and stressed) to take it all in.

General Session: Empowering Modern Enterprise Performance Management – InterContinental - Grand Ballroom B/C

Matt Schwenderman - Partner, Deloitte Consulting LLP; Balaji Yelamanchili - Senior Vice President,

Oracle;

12:15 PM - 1:15 PM

I’m listening to Balaji talk about the development efforts. It’s interesting to note how he states that EPM has been rejuvenated by Exalytics. Things that simply couldn’t finish fast enough now can because Exalytics is Big Iron. How cool is that?

And then the other big keyword for this conference – Cloud. Those of you who have followed this blog know that I have been a big fan of Cloud resources for development, training, kicking the tires, etc. It’s nice to see Oracle bring EPM into this space in an official way.

While me using John Booth’s excellent 11.1.2.2 Amazon image is fine for all of the above, I simply don’t have the expertise or the desire to help clients manage a Cloud instance of EPM. It’s very interesting to hear that Oracle are diving into this both for Planning and Financial Reports. I look forward to telling clients, “Hey, your infrastructure has a bunch of problems. Until your IT department pull their fingers out, how about we start doing POCs on a real, honest to goodness, Oracle environment?” It at least ought to provoke some interesting conversations. :)

Shankar is now demonstrating Planning mobile – wow, this is cool stuff. Yet Another Reason why I can allow my internal Chancellor of the Exchequer to go out and buy an iPad. Wow, that was cool – a really nice bit of UI there.

What’s next?

Mobile and Web-Based Reporting for Oracle Hyperion – InterContinental - InterContinental Ballroom A

Timothy Kosyjana - Finance Transformation Manager, Apex Tool Group, LLC; Mike Williams - Regional

VP, CXO Solutions;

1:45 PM - 2:45 PM

So this is bit of a jolt – as far as I can tell this will be CXO Cockpit’s mobile solution, not Oracle’s. Regardless this should be interesting.

Yep, this is a vendor product session. As always with third party tools, there’s always the risk that Oracle will move into the same place. Looking at what they are demoing, I would say there is some similarity with what CXO offers and OBIEE. I can’t say how Cockpit really compares against OBIEE’s dashboarding functionality but I will note that they claim a two week implementation time, presumably after analysis and design (and of course the databases behind the scenes) have already been completed.

Interestingly, it looks like MDX is the calculation/query engine behind Cockpit on top of Enterprise/HFM as pulled into Microsoft Analysis Server (MSAS). Essbase is a direct connect. IMNSHO, they need to point Enterprise/HFM to Essbase, but I am a wee bit of an Essbase bigot.

Oracle Business Intelligence Enterprise Edition Integration with Oracle Hyperion Datasources – InterContinental - InterContinental Ballroom B

Venkatakrishnan Janakiraman - India Managing Director, Rittman Mead Consulting Pvt. Ltd.;

3:15 PM - 4:15 PM

I’m sitting in on Venkat’s session on Essbase (not really the rest of the “Hyperion” stack) within OBIEE.

I have to say that I run hot and cold and then hot and then…well, you get the idea, on OBIEE against Essbase. I know that a ton of development time and thus money have been thrown against the integration of the two tools so it is an important product direction. I also know that while it is no big deal (so people tell me) to use Essbase in OBIEE sitting in this session hasn’t totally convinced me of that.

Maybe a better way of stating this is – I need to attend an OBIEE/Essbase session that is told from the perspective of an EPM, not an OBIEE geek. No disrespect meant towards Venkat meant whatsoever (I think quite highly of him), but I think I need this stuff spoonfed to me. I think I just admitted I am stupid but I am always ready to speak the truth. All (sort of) kidding aside, I don’t know if I have ever seen this kind of presentation given by an EPM person. It would be awesome to do so.

Hmm, Natalie Delemar tells me that yes, this very thing was done at Kscope13 and I missed it. Oh well, I'll just have to figure it out on my own,

Oracle Essbase New Features and Roadmap Update – InterContinental - InterContinental Ballroom B

John Baker - Director, Analytics, Oracle; Gabby Rubin - Sr. Director, Product Management, Oracle;

4:45 PM - 5:45 PM

Oh this was an interesting one. I am going to do a special blog post on this one because there is just so much going on. I think I may even be allowed to talk about what Gabby and John presented on.

One caveat – everything but everything they talked about was prefaced by the standard safe harbor statement which I equate to “Everything but everything we say is a complete and total lie. But then again, maybe it’s the truth. Your best bet is to listen to what we have to say, think about it a bit, and then just wait till the product actually ships because that and only that is a true indication of what’s in the product.”

Watch this space for more. Or go on Twitter and see what people are saying. I’m going to try to have a more comprehensive approach.

ODTUG Happy Hour – Le Charm Bistro

ODTUG, selected guests, and even me

6:00 pm – 7:30 pm

The ODTUG Happy Hour happened again, happily.

Not so happily, I didn’t wear my black ODTUG shirt. I was a bad board member. Again. I fear I can be kind of a bit of a disappointment.

Here’s the happy black shirt clad Board of Directors having some truly delicious food and drinks.

Well, at least I see Tim Gorman in his proper uniform. That’s an oopsie on my part. Sorry, ODTUG.

And here I am in, not quite in the Uniform of the Day, almost getting the future scoop, on paper, of one of the new features in Essbase that I will (hopefully) post tomorrow. Can you believe the Essbase PM literally snatched the paper out of my hand. Yeah, I can, too.

So close. But no cigar.

Be seeing you.

Continuing the coverage, with a break in the beginning

No, I did not sleep in, nor did I over imbibe last night (for the record a glass of soy milk and a cup of tea – I am so boring), but instead I went to the Oracle Publishers Seminar, courtesy of the Oracle Publishers Program. And I got into that because of OTN’s Oracle ACE program. Whew.

Did I actually do anything to deserve being there? I did, so long as one remembers that I had 12 coauthors and editors that all helped make Developing Essbase Applications possible. I guess since during the writing of that book I whined the most I get to attend sessions like this. All kidding aside, because the ACE program got me access to Oracle’s Publishers Program, I was able to be the editor in chief of DEA and that’s really the reason I was there.
I’d love to tell you more about it, but all of the cool product futures are under NDA. I did get some very pointed suggestions re books about Business Analytics and I hope to incorporate at least some of them within the second edition of DEA.

What’s New and What’s Coming: Oracle Hyperion Enterprise Planning Suite – InterContinental - InterContinental Ballroom C

Shankar Viswanathan - Director, Product Management, Oracle

Lots of cool stuff coming in Planning. The highlights are, in outline format and please note that while this is not NDA’d always, always, always KNOW THAT FUTURE FEATURES MIGHT HAPPEN AND THEN AGAIN, MIGHT NOT. In other words, everything Oracle stated was under safe harbor, i.e., they are thinking about this stuff BUT NO PROMISES and what they present MAY NOT MAKE IT TO THE PRODUCT and the performance MAY NOT COME CLOSE TO WHAT IS PRESENTED. Are you getting this? Don’t make buying decisions on future items unless Oracle tell you otherwise. This blog would not be the place I would risk the price of a cup of coffee. And I wrote it. With those hopefully strong enough caveats and qualifications here’s what’s coming in Planning :

What’s new/roadmap

PBCS
Planning mobile
EPM mobile approvals
UI & usability improvements

Post 12 months

Cloud module

Continued ease of use
PFP and PSPB in PBCS

Analytics from Planning

OBIEE Reporting integration
Tighter FR integration

Missing

Full data copy for ASO to BSO and vice versa including relational functionality
Trickle feed
Sandboxing

Out of box contents

WFP will be more global
Functional starter kit to help kick start projects

Enterprise planning

EBS Projects integration with PFP
Primavera P6 with PFP (not sure about that last bit)

Cloud

Planning on Oracle public cloud

Pure SAAS deployment

No need to maintain it
Much faster implementation cycles

Application diagnostics

Diagnose design before rollout
Show warnings and errors to make performance efficient
Runs on entire app or selected bits
Good design practices at design time but in an automated fashion
Bad things blocked at run time
What’s monitored/blocked/governed

Dimensions
Business rules, in forms and standalone
Forms
Planning Units
Map Reporting

Planning Mobile

Mobile browser
Optimized for mobile
Key end user functionality

Smart View for Win 8 Pro

Office 2007-2013
HSF and SV for Planning

Don’t forget: NONE of the above may happen. Or all of it might. Then again, maybe not. There’s no way to tell and even Oracle don’t know what might or might not really get delivered. So if you make a single strategic decision based on what I wrote above…well, you’re even dumber than I am. Which is saying something. So take the above with a huge grain of salt and a very general guide to future direction. And enjoy.

Oracle Business Intelligence Product and Technology Roadmap

Paul Rodwick - VP, Product Management, Oracle;

It’s interesting, or maybe I should write it isn’t so surprising (which, I grant you, is sort of the opposite of interesting) that so many of these slides are the same. Not a criticism of Oracle but more like presentation proof that the EPM and BI world are merging.

I think I love it.

Essbase/EPM Meetup

7:00 to 10 pm

Here’s all you need to know:

The best Essbase meetup at OOW 2013. 24 September, 7pm, Specchio-Ristorante

I’ll try to have pictures again.

Be seeing you.

Oh noes, we’re halfway done

It is sort of scary how fast conferences go by. Maybe I shouldn’t be so shocked but wow, attending OpenWorld feels like flying a Super Étendard about 50 feet off the deck at 600 knots. Exciting, in other words.

OpenWorld is a very exciting time, because of all of the glitz that only a firm the size of Oracle could bring and because of the fantastic information sharing and networking opportunities that a conference of this size affords.

So, will Developing Essbase Applications ever have a second edition?

The answer is yes, exact contents to be determined. I had breakfast with my editor of CRC Press at David’s Delicatessen and it is a go. Now it’s just up to We Happy Few to figure out what the heck we are going to do with the second edition.

We’ve done well as far as sales goes, but the Essbase world is changing, and we need to make the book mirror that new landscape. I have a bunch of ideas (probably too many) and I need to work through what will be new, what will be revised, and what just might get dropped.

I’ll see how many new grey hairs will result.

Oracle Enterprise Performance Management in the Cloud

Ananth Avva, LiveOps Inc.; David Pooley - Sr. Director - EPM Global Market Development, Oracle;

Jennifer Toomey - Senior Director, Business Analytics Product Marketing, Oracle;

I’m sitting in on the cloud presentation given by Oracle and my buddies at Qubix. As readers of this blog know, I’ve been a huge fan of EPM in the cloud for development purposes.

On my last project the client suffered a number of infrastructure issues and we were able to go to the Amazon cloud to run EPM 11.1.2.2 without issue and then migrate back down to their servers when they were available.

But of course I couldn’t guarantee security (I am an Essbase hacker, not an infrastructure hacker) so we very purposely did not bring any real data up to the cloud, did not use any employee metadata (which was a bit difficult as there was a workforce application), and in general tied our hands. It was better than nothing, in fact a lot better than nothing, but we were constrained in what we could do.

I’d love to be able to go to the client and say, “Hey, Oracle’s got a product, it’s the full deal, and it’s secure. Let’s get that going for the first phase and the project timeline pressure that hinges on your infrastructure simply goes away.” Yes, that sure would be nice. And now it’s possible.

Roger Cressy of Qubix is now speaking. His comments:

Not just Planning in the cloud but big rewrite of the tool.
Biggest change to the tool since it came out.
Customer benefit is that the dependence on infrastructure at project startup is simply gone, no install, opex vs. capex, and this is full Planning, not a crippled version of the tool.
Consulting doesn’t go away (Cameron the consultant is glad to hear this) so functional requirements and design is still there. Data and metadata extractions, optimizations, calcs, etc. are all still part of the Planning implementation game.
But from both a customer and consultant perspective, the big dependencies on infrastructure are gone.

Real-World Performance: What Is Big Data, and What Makes It Big?

Andrew Holdsworth - Senior Director of Real World Performance, Oracle; Kaiyao Huang - Principal

Performance Engineer, Oracle; Michael Hallas - Architect, Oracle;

Trillion row tables? Hmm, a bit bigger than your typical Essbase data source. An interesting bit of math is: if you process a row a second, how long does a trillion rows take? Answer: 32 years. Hmm, I have seen slow queries but that would most definitely take the cake.

If I were going to try to summarize this session (which apparently is part three of three), it is how to use set-based SQL along with a bunch of other good SQL practices (which are mostly over my head, sadly) to tackle really big data sets.

Product Development Panel Q&A: Oracle Hyperion EPM Applications

Matthew Bradley - VP, EPM Development, Oracle; Prasad Kulkarni - Senior Director, Financial Planning

Applications, Oracle; Kash Mohammadi - VP of Development, Oracle; Douglas Cosby - VP of Software Development, Oracle; Rajesh Bhatia - VP of Product Development, Oracle;

I am looking forward to this one. A lot.

And I wasn’t disappointed. Oracle took all questions with quite a bit of grace – I wouldn’t want to be the St. Andrew of product management. Nothing was off the table and I for one am glad that Oracle are as open as they are.
I asked the question I seem to always ask at meetings like this – when oh when oh when will we get METAREAD filters in Planning. The answer is the 32nd day of Never, unfortunately. The Way Forward is to use Planning data sources and I suppose that makes sense when you want to capture all of the relational goodness and Smart Lists that Planning supports.
So for those of you who persist (like me, for instance) in preferring Essbase as a data source, may I point you to this post here on rolling your own METAREADS from Planning.

There’ll be more

I’ll be (hopefully) doing a better job today with the live blogging.

The Essbase blog post to end all Essbase blog posts

That phrase could mean two different things:

I am going to share with you some of the really cool things that Oracle have announced at OpenWorld and so this is just a turn of phrase to show how excited I am by all of this.
I am going to share with you some of the really cool things that Oracle have announced at OpenWorld but maybe didn’t want broadcast to world+dog and by doing so I will royally annoy Oracle product management, development, and probably areas of Oracle (hint: legal) I’ve never heard of, get kicked out of the ACE program, and in despair I give up blogging.

Hopefully this post will fall more into the former than the latter but one never knows. In my possible future defense, I note that there are a bunch of Tweets out there that cover some of this very same stuff and there were people taking pictures at the session, so maybe it’s all okay. I hope.

So why bother?

I didn’t take any pictures (I got in trouble last year but happily my picture taking skills are pretty bad) this year although I saw many doing just that.

What I did take were notes. Somewhat alarmingly, I am actually not half bad (note, I did not say I was good) at typing thanks to my much-missed grandmother who taught high school business in Royal Oak, Michigan. And that meant when we visited her in the summer (Michigan is a great place for a kid) we got to play with her electric typewriter (yes, it was really cool in the 1970s and yes you now have an idea about how old I am) and all of the typing books publishers sent her as samples. And that meant that in those long ago days before video games, I learnt how to touch type. And that means you now understand why my blog posts are so long. Whew.

So with the caveat that I am on the outside looking in, and with the additional HUGE caveat that everything Oracle stated was under safe harbor, i.e., they are thinking about this stuff BUT NO PROMISES and what they present MAY NOT MAKE IT TO THE PRODUCT and the performance MAY NOT COME CLOSE TO WHAT IS PRESENTED I think I can now relate what Gabby Rubin and John Baker showed to the almost completely sold out room.

What’s new in 11.1.2.3

Yes, this isn’t exactly the future as 11.1.2.3 is here now, but Oracle related some DEVELOPMENT LABS numbers that I thought were pretty exciting. What does Oracle mean by DEVELOPMENT LABS numbers? DEVELOPMENT LABS numbers mean: in development code lines, using incredibly specific test cases, on servers of unknown capacity, such and such results were observed. What I take that to mean is that these numbers are a rough guide to performance but for goodness’ sake, do not take this as gospel – as always with Essbase, your performance may vary quite a bit and that variation may all be on the bad side so test, test, test. And don’t say I didn’t tell you so when it all goes pear shaped.

With that caveat, here are the big things in 11.1.2.3:

New BSO functions, especially @CREATEBLOCK and run time Essbase Substitution Variables
ASO dimension builds are improved (no percentage given), especially with shared members
Better MDX performance
XREF performance is hugely improved, sometimes coming very close to intracube performance. NB – These are DEVELOPMENT LABS numbers. Test, test, test.
The JAPI has been improved but honestly my highly-touted note taking skills failed me here.

So the big one in 11.1.2.3 is the XREF performance. That means to me that all of those games we play in Planning-land to get XREFs declared as stored (and taking the cost to do the data moves) may no longer be required. And that we can reevaluate data export processes in all kinds of BSO databases to move data across databases – maybe file exports aren’t required any more.

What’s in the future for Essbase, post 11.1.2.3

A bit of background

Essbase is in kind of a unique position vis-à-vis other products – it straddles both EPM (my home) and Business Intelligence (my home, one day, maybe). In some respects this is fantastic for those of us who love Essbase because that means it gets funding. Of course there is a flip side that means that functions are sometimes for BI and don’t come over to EPM because it simply doesn’t make sense. At least Oracle thinks it doesn’t make sense where you or I might disagree.

An example of a BI-only function in Essbase is its Essbase Persistent Aggregates – these are totally dynamically generated Essbase databases that are optimized for performance against OBIEE.

And there’s another bit about Essbase that I think most of us aren’t aware of – Essbase is the analytic engine behind Oracle Fusion GL Balances, CRM Territory Management, and Projects Performance. These aren’t Essbase databases we’re supposed to access via Smart View and in fact may make our brains explode as we bring our EPM perspective to how a cube ought to look. Gabby made an analogy that stated that what we might think of as a four dimension model may resolve to 14 or 200 dimensions in Essbase. Yup, that sure isn’t Sample.Basic.

The next 12 months

Again, remember, this is all what Oracle product management would like to do. They don’t know if it will happen. I don’t know if it will happen. Under no circumstances make any business or technology decisions based on what I am going to write.You Have Been Warned.

With that yet again restated absolutely huge, did-you-pay-attention-this-means-you caveat, here’s what I at least think I heard.

Restrictions

Naming restrictions like member name length, reserved keywords, special characters – all of that is a thing of the past. Before you go crazy with joy, remember that when your relational source has crazy ascii characters they now are going to come into Essbase. I am thinking of the Biblical giveth with one hand, taketh away with the other. Maybe a better way to think about this is that flexibility can be a double edged sword.

This will be available first in JAPI and SQL data sources. The whole issue around uniqueness will be addressed at a later date.

Performance

Improved performance for queries, calculations, and API performance are all on the roadmap.

Performance (and I guess to some extent features) are going to have a bit of a split between Exalytics and everything else. When a terabyte or two or more of RAM along with 40 or 128 CPU cores are available to Essbase, Oracle know that they can do certain things. When there’s 32 gigabytes of RAM and four cores, Oracle realize that there is much Essbase simply cannot do. That difference in capacity is going to tell, especially with the continuing success of Exalytics. They’re not abandoning the non-Exalytics world but those of us in the “normal” world will just have to look over the wall and sigh in envy.

I mentioned above that Essbase is being driven by BI, EPM, and Fusion. You will see functions and performance come out that may only make sense in one of these environments. This is going to be weird for those of us who have used Essbase since the beginning but again just a fact of life.

Simplified sophistication

Oracle know that we Essbase geeks love to twirl the Essbase dials. This is particularly true in BSO but there is bit-twiddling that is available in ASO as well. That’s all well and good for those of us who enjoy it but their goal is to take the sophisticated tuning we geeks do and make it available to everyone while not losing that option to do the tweaking of the settings. I for one am very curious to see how this works out. Hopefully no more brain-dead Essbase databases will be the result.

Enhanced calc script language

The BSO calc language that we all know and love is viewed as a real product differentiator that makes Essbase the awesome thing it is. Note that this is not MDX. More on this in a bit.

As the BSO calc language is cool, awesome, and generally rocks, it will continue to be enhanced.

FIXPARALLEL

If you think about what needs parallelization in Essbase, aggregations are an obvious choice and of course it already exists. But what about all those calcs that use FIXes to “do stuff”? That will be, someday, available as a parallel process as well.

Beyond just member formulas and allocations, DATACOPY, DATAEXPORT, XREFs (this is beyond 11.1.2.3’s better performance), and XWRITE all will work within FIXPARALLEL.

FIXPARALLEL will work within FIX statements and vice versa. Btw, I wrote this when I heard it: “That is pretty damn cool.” And it is.

Additionally, you will be able to control task dimensions for a division of work that is not dependent on outline order and when specifying the parallelism you will be able to define the number of threads. Within those threads there will be new Essbase variables (a feature I use all the time today) that are initialized within each thread and for each task within a thread.

XRANGE

XRANGE will be, maybe, extended to many more (approximately 30) functions but I couldn’t type them all fast enough in my notes. So we’ll have to wait and see what happens.

Smart View Recorder Services

This is an internal utility that Oracle will provide through OTN so now there will be no support. Essbase Hackers aren’t afraid of that, are we?

Recorder Services is an Application Provider Services add on (I have no idea how it is installed) that can be used as a performance and data benchmarking tool. It records user activity in APS from Smart View and logs it to…disk? Tables? Sorry, I missed that part but I am guessing to disk.

That log can be analyzed or replayed. Imagine what you might do with that. <insert big grin>

MDX improvements

MDX is not the red headed stepchild that is beat on a regular basis. Nope, it’s the key to ASO Essbase and of course is a pretty awesome query language for ASO and BSO.

So what can we expect to see as improvements to MDX? Better optimization for attributes (Does this include varying attributes aka Slowly Changing Dimensions? Dunno.)

Optimized Aggregate command is coming. This is very commonly used in BI queries and there will be significant performance improvements for totals at multi-level hierarchies. Improvements will be based on query and dimension depth.

The big MDX improvement is the concept of Sub Selects. What this means is that in order for Essbase to support the huge Essbase cubes that BI generates, Sub Selects slice out or filter portions of the ASO cube and force operations to only apply to that subset of the overall database. This means that relative performance to the whole cube can be realized. What I take that to mean is that less data equals faster performance. When the required data set for a particular calculation isn’t the whole ginormous ASO database, this could be a very valuable function indeed. Note that relative performance will seem greater the larger the database in question – again this is a case of limiting the scope of the calculation.

Query parity

There is a big difference between data results from SQL versus Essbase. I think of this as the top down nature of SQL versus the bottom up nature of Essbase. By that I mean – for a given data set, if the Essbase outline doesn’t map to all of the fact members, data will get rejected (hello dataload.err). For those of you who read Developing Essbase Applications you will note that I wrote a chapter that forced metadata matching to fact to get round this very issue using Essbase, ODI, and SQL.

Oracle have taken a different approach with what they call the Renegade Member. Instead of forcing the metadata to match the fact, they create default members that get the aggregated results of all of the non-matching fact data. This is huge, to put it mildly as totals shall now always tie.

Of course you are still going to have to figure out what’s missing, but in fact there will be a Renegade Member error log that you can then use to build hierarchy.

You will be able to have one for each dimension if you wish (I can think of some dimensions where it would not be terribly likely) and the suggested good practice is to stick this Renegade Member at the second generation of the dimension.

The big one – Hybrid Essbase

I love BSO Essbase. I love ASO Esbsase. Sometimes BSO is the answer, sometimes ASO is the answer, sometimes it is a real stinker trying to bring the two of them together.

Wouldn’t it be nice to have a database that gave us the flexibility and power of BSO Essbase with the raw power of ASO? That would mean we could continue to do all of the cool block-y things we do in BSO and then not worry so much about what happens when BSO Essbase blows up when the database gets too big.

What the Hybrid Essbase engine does (Slight digression: ASO, BSO, so obviously CSO?) is give you a choice about aggregation. At the base is always the same level zero blocks that BSO Essbase has always supported. Above that, with Hybrid Essbase (HSO?) you have the choice of making some dimensions dynamic (or even all of them if you chose so) or some dimensions dynamic and others stored (so same BSO aggregations with blocks).

This is not going to be like ASO so no Multiple Hierarchies – the dynamic nature is either on or off on a dimension basis.

What is also not like ASO is there is no ASO bitmap but instead (as I sort of understand this) the very best Intelligent Calc (or something akin to it) the world has ever seen. Hybrid Essbase will understand what you are looking at an upper level and will sum the blocks dynamically and do that aggregation very quickly indeed.

You will still have the option of stored upper level members and that will act like a BSO stored dimension today.

Using DEVELOPMENT LABS NUMBERS, Hybrid Essbase pretty much matched ASO Essbase and in some instances beat ASO performance, at least in the case of one development test case. Who knows what the real world will bring but the numbers were very, very, very and did I mention very impressive. How do they do this? Beats me and yeah, when you don’t understand something well it seems like magic. :) I love magic shows.

Just to make this clear (and to reiterate why this is so exciting): Hybrid Essbase has BSO capabilities with ASO dynamic performance. I can hardly wait.

Beyond the next 12 months

Gabby also related what might come out in Essbase beyond the next 12 months but it was so much, so fast, and so tenuous that it simply isn’t worth listing.

Also, that is a teaser to get you to come to Kscope14. :)

And that’s the end

Hopefully just of this post, not my blogging career. The new features are just so exciting I thought I owed the Essbase world at least a peek into what is coming from Oracle, maybe.

I have to say that I wrote this post with some trepidation but I hope my frequent OMG DO NOT BELIEVE ANYTHING I WRITE statements will give you some idea of what’s coming for Essbase while not committing Oracle to much of anything.

Some of this (actually, a lot) was available at Kscope13 but it was super tenuous and Oracle specifically asked us not to write about these future features. The restrictions seem to be (mostly) off and I for one am hopeful that many if not all of these features actually ship, and soon. But I’m not holding my breath and neither should you.

Be seeing you.

It couldn’t be done, but has

And apparently by someone quite a bit more inventive than me just playing around. Who would have thunk it? Not me. In fact, quite explicitly not me, as I have told multiple clients (I think two, maybe three) that this very thing cannot be done. Sigh. But thanks to this thread on Network54, it most definitely has been done. All thanks must go to Christian M. (who oh who oh who is Christian M.? Christian, if you’re coming to Kscope14 I want to buy a beer. Several, in fact.) as he figured this out. So this is most definitely not something I have invented – I am just expanding on Christian’s discovery.

Why this matters

Yes, I know, I haven’t exactly spilled the beans on what this is, but let’s set some background so you can understand why this is so important.

The technique that I call Focused Aggregations is a way to read a Planning form and only aggregate the bits of the hierarchy that are germane to the form. Calc Mgr/HBR can read POV and dropdown values from Run Time Prompt (RTP) variables and “faking” the aggregation process (really, it’s just aggregating as if a CALC ALL or AGG (dimnames) was issued but it only does the aggregation for the relevant hierarchies all the way to the top of the dimension). This approach is fast because instead of aggregating an entire Entity or Product or Employee or whatever dimension, much of which is not going to be relevant to the data entered on the form, only the hierarchies that matter get aggregated to the dimension top.

This approach is fast, fast, fast (it isn’t magic, but only aggregating the bits you need versus an entire dimension can make “too big” BSO Planning applications perform acceptably) and I have written about it here in Hyperion Business Rules, Calculation Manager, and Dodeca. The Dodeca post is (or was) an important one within the field of Focused Aggregations because focused row-based aggregations are possible as Dodeca can provide the row contents. That’s what a Planning form cannot do. Or at least it couldn’t, until now. Hopefully you now understand why I am so excited about this.

Why Calculate Form isn’t good enough

Some (like Celvin Kattookaran, my coopetition on this subject) have suggested that maybe the in-built <Form Calculate> might do the same thing and then why oh why oh why would anyone bother with focused aggregations? Here’s why this approach doesn’t make the grade and Celvin mostly hit on the reason why in his post.

A bit of review

Just in case you’ve forgotten what a <Form Calculate> Business Rule looks like, here it is in 11.1.2.3’s form designer. Note that this is not a rule that you can look at (maybe it’s buried in the EAR file that makes up Hyperion Planning but it is not something you can casually examine).

Note that I’ve moved the <Calculate Form> rule over to the “Selected Business Rules” listbox and ticked the “Run on Save” box.

Going into Smart View, I can open up a form (note the dynamic user variable selection of E032 – that will become important a little later on), and enter in values of 100 and 200 into the January column.

When I click on Submit, I see the following in the form:

Very nice, right? There’s 300 in E032. Is this enough? Nope.

The problem with Form Calculate

The issue with Form Calculate is that it only aggregates the descendants of the topmost member in the form. As E032 was the form’s top of the Entity dimension, all of the intermediate members were aggregated. That is a Good Thing. But what Form Calculate does not do is aggregate the ancestors of the topmost member in the form. Note that E03 does not equal 400 (and it should). And TotalGeography does not equal 1,173 (ditto).

Here’s what the totals ought to look like:

So unless you live in some weird Planning world where looking at top level numbers isn’t important, I’d say that Form Calculate is nice, but not enough. Remember, for anyone looking at data above E032, it’s as if the Form Calculate didn’t happen.

The traditional way of aggregating the form dimension

It’s really simple:

But remember, Entity could be a really big dimension with lots of lower level blocks and thus pretty slow performance. Are you sure you want to aggregate that entire dimension?

Enter Christian M.’s most awesome hack

What would be nice would be a way to read the dimension, or even more to the point the topmost form member and then aggregate the descendants (like Form Calculate) and the ancestors (which Form Calculate cannot do). Until now, this is what could not be done.

Christian’s hack still can’t read the Planning form’s row definition, but what it can do is read a Planning User Variable. And if a form’s row definition is driven off of a Planning User Variable, then you have squared the circle because that Planning User Variable can (somehow) be read into Calculation Manager. Huzzah! Hurrah! Hot diggity-dog, etc.

Here’s the technique in summary:

Create a Planning User Variable
Create a Planning form that uses the User Variable to drive the row contents
Create a Calculation Manager Variable that is explicitly assigned the Planning User Variable

The Calc Mgr Variable type must be Members, not Member, and no, I don’t know why, and yes, Members versus Member doesn’t make sense but why are we complaining about a great hack?

Use the Calc Mgr Variable in the standard Focused Aggregation code approach
Deploy the rule and assign it to run on save in the form.
Hope that Oracle doesn’t “fix” this hack.

Let’s now go through this in detail.

Create a Planning User Variable

Simply go to the Administration->Manage->Variables and create a new variable. I like to sort of mnemonically name them, so my Entity dimension variable name will be varUVEntity.

Create a Planning form that uses a User Variable

I simply modified a form in the Planning sample application and in the row definition selected the descendants of the user variable:

The My Region variable comes with the sample application. I don’t like spaces in variable names and I don’t like variables that aren’t immediately identifiable as such, hence the name “varUVEntity”.

I wanted the user to be able to change the row contents, so I made sure that the user variable was selected in the Other Options form design tab and I selected “Enable dynamic user variables”. That last tick box means that the user can change the form rows as required so long as he stays within his Entity dimension metadata security.

So far all of this is bog-standard Planning. We’re about to enter the undocumented hack area.

Create a Calculation Manager Variable that reflects the Planning User Variable

This is the bit where I really have to take my hat off to Christian. I am pretty sure I would have come up with this approach in, oh, never.

Go into Calculation Managers Variable Designer and create a variable of type Members. Assign (you are going to have to remember the name of your variable as it isn’t going to pop up in Calc Mgr) the name of the Planning User Variable to the Calc Mgr Members variable’s Default Value. Be sure to prefix it with a “&” symbol. And yes, that is sort of like an Essbase Substitution Variable, but isn’t.

You will also need to select a Dimension Name (in this case, Entity), make sure it’s a RTP type, and even enter in a RTP Text message. Remember, it’s the Default Value with an ampersand before the Planning User Variable that does the passing of the Planning User Variable’s value to Calc Mgr.

NB – It seems logical to me to use a Calc Mgr variable type of “Member” instead of “Members” and in fact it works, but when using that variable type a web browser will force a dialog box. Oddly, Smart View does not do this.

Don’t try to find reason where I suspect none exists – none of this makes any sense. I’ve done the pig headedness bit for everyone, so just go with Members.

Write a Focused Aggregation Business Rule in Calculation Manager

Calc Mgr script

Please disregard the lack of variables in the FIX statement – you can add that in later. The important bits are the @IDESCENDANTS and @ANCESTORS statements.

In the first sub FIX statement, the code fixes on a Calc Mgr variable called varSegments. This is just a standard RTP Calc Mgr variable.

What’s inside the FIX({varSegments}) is what’s important and heretofore impossible. The code does an @IDESCENDANTS of the Planning user variable varUVEntity as passed to the Calc Mgr variable varBRUVEntity to mimic the Form Calculate functionality. The next line issues an @ANCESTOR calculation of that Planning User Variable via a Calc Mgr variable to aggregate up to the top of the dimension. Ta da! We have just hit the super cool hack.

The second FIX statement uses that same Planning User Variable as expressed in a Calc Mgr Variable but now selects the relevant bits of the Entity hierarchy and then does an @ANCESTORS aggregation of the Page dimension Segments.

Double Ta-Da! We have just squared the row-based Focused Aggregation circle.

Calc Mgr rule

Stick the script into a rule by dragging and dropping it into the rule itself.

NB – If you do not have Flash 10.x installed on your machine (in my case, a Windows 2008 R2 VM) you won’t be able to do this.

When you do this, be sure to go to the Variables tab and tick the “Is Hidden” boxes for the variables. This will ensure that neither RTP pops up when the rule is run on form save.

If you click on the Script tab you will see something kind of interesting:

Instead of varBRUVEntity the default value, &varUVEntity, which does look awfully Essbase Substitution Variable-ish, shows up.

If in fact you were to go to the Errors & Warnings tab and have Calc Mgr analyze the script, you would see this bogus warning:

Interesting, isn’t it? But it still works.

Deploy the rule

But don’t validate

As Celvin pointed out in his blog post, don’t bother trying to validate the rule. That error message in the Errors & Warnings tab will surface:

So just deploy. And fix the code yourself. C’mon, if you’re writing code like this you can do syntax checking in your head, right?

Deploying is easy

Hook it up to the form

We are back to bog standard Planning. Simply attach the deployed Calc Mgr rule to the form:

As always, make sure it runs on save, uses the members on the form to drive the POV/Page dimensions, and hide all RTP prompts. That’s it. Let’s now go prove that it works.

Hmmm, I love pudding

The proof of the pudding is in the eating.

Of course we cannot eat a Planning form, but we can:

Change a dynamic user variable
Enter in data values
Save the form
Prove that it aggregates all the way up to the tippy-top of the dimensions

Let’s do just that.

Change a dynamic user variable

Select E01_101 as an Entity member.

Clicking on OK and then doing a form refresh will change the form selection:

Enter values and Submit Data

Review the fully aggregated data

And here’s the data aggregated all the way up to the top of the hierarchy in an Essbase ad-hoc retrieval:

Magic. Thanks again, Christian.

So why is this faster?

It’s all in the amount of data that Essbase has to slog through in its aggregation.

I turned SET MSG DETAIL on in the script (and got rid of the CALC DIM of Accounts and Period as those should be fully dynamic – ask Oracle why the sample app isn’t set up that way, not me):

[Tue Oct 15 12:10:53 2013]Local/SampApp1/Consol/hypadmin@Native Directory/1792/Info(1012672)

Calculator Information Message:

Total Block Created: [0.0000e+000] Blocks

Sparse Calculations: [2.2500e+002] Writes and [8.5500e+002] Reads

Dense Calculations: [0.0000e+000] Writes and [0.0000e+000] Reads

Sparse Calculations: [1.0303e+006] Cells

Dense Calculations: [0.0000e+000] Cells

[Tue Oct 15 12:10:53 2013]Local/SampApp1/Consol/hypadmin@Native Directory/1792/Info(1012550)

Total Calc Elapsed Time : [0.059] seconds

I then created a calc script that used an AGG for Entity and Segments:

And ran it with the following results:

[Tue Oct 15 12:11:38 2013]Local/SampApp1/Consol/hypadmin@Native Directory/360/Info(1012672)

Calculator Information Message:

Total Block Created: [0.0000e+000] Blocks

Sparse Calculations: [2.8690e+003] Writes and [1.0772e+004] Reads

Dense Calculations: [0.0000e+000] Writes and [0.0000e+000] Reads

Sparse Calculations: [1.3137e+007] Cells

Dense Calculations: [0.0000e+000] Cells

[Tue Oct 15 12:11:38 2013]Local/SampApp1/Consol/hypadmin@Native Directory/360/Info(1012579)

Total Calc Elapsed Time for [BRAggTst.csc] : [0.434] seconds

Some analysis

Agg	Focused	Variance	Variance Percent
0.434 seconds	0.059 seconds	-0.375	-86.406%

Nothing to sneeze at here – that’s a decrease of 86% in the amount of time the calculation took to run. Worthwhile when applied to your real world application? I think so.

Let’s look at the number of transactions.

Approach	Sprase writes and reads	Sparse cells addressed
AGG of Entity and Segments	2,869 writes, 10,722 reads	1,313,700
Planning User Variable Focused Aggregation	225 writes, 855 reads	103,030

There’s an enormous difference in the number of writes, reads, and overall sparse cells. No wonder a Focused Aggregation is so much faster.

Where do we go from here?

Well, as far as this blog post is concerned, hopefully nowhere. This is quite the post (19 pages in MS Word which is where I draft all of my blogs – that’s quite enough of the Cameron Effect for one day) but I think it was important to showcase how absolutely brilliant Christian’s hack is, and how important it can be for the Focused Aggregation approach which can now finally apply that technique to rows.

I am going to reach out to the Calculation Manager Product Manager (Sadly, I really only have the emails to a couple of people within Oracle – I lay that at the feet of being an independent with just about zero impact on sales. Or maybe I am just lazy and don’t work the contacts the way others do.) and beg, beg, beg that this functionality not be removed. Ever. In fact, it would be super if this functionality was formally incorporated into the tool so we didn’t have to play games with default values and ampersands. I am going to guess most of the functionality is already there within Calc Mgr so I am hoping we’ll see it soon, officially supported.

One last time – Christian, thanks man. You have no idea how happy this has made me.

Be seeing you.

Introduction

Most (Some? None? All?) of you know that I am on the ODTUG board of directors. Of course you, the ODTUG membership are responsible for that and I am obliged for your confidence in me. It sounds a cliché, but serving as a board member really is a tremendous privilege and responsibility.

Board members who care, a lot, are the hallmark of a successful user group and I am pleased to state that your ODTUG board of directors is filled with some of the most caring, passionate, and smart people I have ever met. How can I drive this home? Sometimes, when I meet with people professionally, I think to myself (and yes, I do have the conversations with myself and yes, it worries me too, as it should), “Hmm Cameron, this lot definitely isn’t top drawer, you know more than these bozos do combined.” Other times, I have flashes of clarity that say, “Cameron, you’re the bozo. By a long shot. Try not to embarrass yourself too much.” And so it is with the ODTUG board.

With that little bit of probably too close for comfort humor, you hopefully get the idea: the ODTUG board is made up of really smart people whose service has helped produce the very best Oracle user group extant.

Introducing…

With that, I hope you understand the importance that individuals can make to the board and by extension ODTUG itself.

And that brings us to the point of this blog post – the just-declared ODTUG board of directors elections.

You haven’t kicked us out yet

Four of us did not run for election, although our moment of doom comes up next year. Additionally, two incumbents were reelected. I’m honored to count as my peers:

Tim Tow
Barbara Morris
Tim Gorman
Martin D’Souza (incumbent, reelected)
Monty Latiolais (incumbent, reelected)

And the new ones

But of course new people were elected as well. They are:

Natalie Delemar
David Schleis
Mia Urman

Natalie is a fellow EPM geek whom I met for the first time at Kaleidoscope (as it was then called) 2008 – the very first ODTUG conference that included the EPM community. Oddly, I had “conversed” with Natalie through web forums but didn’t even know her real name as she went by DaveJonJoshMom. That ability to actually physically meet people is one of the things that makes ODTUG so special. Natalie and I went on to work together on the Hyperion SIG, the EPM/BI content selection process, and of course as coauthors in Developing Essbase Applications. I’ve been after Natalie for years to run for the board because I think she will bring great insight and energy to the job. I am therefore beyond pleased to see that she made it and will help represent the not entirely small EPM/BI Kscope contingent at the board level.

David actually has been on the board before as an appointee for a board member who did not complete his term so I have worked with him before. He is equally passionate about the technologies he represents and additionally has an atrocious taste in shirts (wait, is it me who commits the fashion faux pas?) that simply must be admired. David also has a sense of humor that embraces the ridiculous and is an engaging speaker. In short, I am really looking forward to working with him again.

I’m afraid that I don’t know Mia, but my ignorance is in no way a reflection of her qualities. A quick google-stalk of Mia shows that she is the president of her own firm, an Oracle ACE, and an expert in ADF and Forms. In short, she is the typical highly accomplished ODTUG board member (I am overlooking yr. obdnt. srvnt.’s somewhat inexplicable inclusion in that august group of geeks). I look forward, a lot, to working with her as a board member.

The ones that are going away

Lest you think that all of my Sammy Davis Jr.-like encomiums are mere window dressing, please know that I am beyond sorry to see Bambi Price, Jerry Ireland, and John King roll off the board. All of them were (and are) fantastic contributors to ODTUG. I know that they will remain active in ODTUG.

This is what democracy looks like

You, the ODTUG membership (only full ODTUG members can vote – if the above election fills you with joy, great, if instead you are filled with a rage inchoate, next year’s election is your chance to throw we bums out, but only if you are a fully paid member) are responsible for the results of the election. The board represents your interests, your needs, and your technologies. I hope you are pleased with the results of your voting – I know I am.

Be seeing you.

Introduction

This post, I’m afraid, will be about frustration. Specifically, how Excel ate a day of my life, and project timeline, and project budget. And also how Excel added yet more grey to my head (oh well, at least I have hair). Read on, gentle reader, and yr. obdnt. srvnt. will show you how to avoid Excel driving you nuts.

Introduction to the introduction

My saga of frustration, wonder, and pain all came about because of a simple request to load data from an Excel workbook into Essbase. A lock and send (this was to a BSO cube) is amongst the most basic tasks we can do with Essbase. And I managed to get it wrong. Sigh. Some days are just like that, right? But the strange thing was that I tried really, really, really hard not to get that send of data wrong. In one of those cruel and ironic twists of fate, that very act of trying to get it right is what doomed me. Like I wrote, it was just one of those days.

Some Excel functionality you may not be aware of

Just about everyone knows how linked Excel formulas work. For the one or two of you who do not, here’s a recap of this supposedly straightforward process. Don’t be put off by the simplicity of the examples – it illustrates the issue quite nicely.

Step 1 – create a source workbook

Easy peasy, lemon squeezy. This is about as simple as a file can get with a single cell of data and a label to identify it. Oh, Source.xls, I love thee.

Step 2 – Create a target workbook

It couldn’t be simpler, right? Create a formula and link cell C2 in Target.xls to C2 in Source.xls. Oh Target.xls, I love thee too.

Step 3 – Create a second target workbook

Huh? Why would anyone do this? Well, in my case because multiple workbooks linked to the same source file is highly germane to this Frustrating Excel “feature” so please humor me. And really, it isn’t that unreasonable to have multiple target workbooks linked to the same file. So Target_2.xls, love is still in the air.

Now we have three files, all with the 762 value. What oh what oh what could the big deal be?

Step 4 – Close one of the linked workbooks

This couldn’t be easier – simply close Target_2.xls. Now there are two files: Source.xls and Target.xls open, both still with a value of 762 in cell C2.

Pay attention, pay attention, pay attention

This is important: Target_2.xls also has a value of 762 in cell C2 as linked data in Excel is stored in two ways, both as a formula link and as a local value. I know that’s confusing, but that feature is how Excel can display information in target workbooks when the source isn’t available.

Stick with me folks, it is about to get very interesting.

Step 5 – Change the value in the source workbook

In Source.xls, change cell C2’s value from 762 to 303. Now both Source.xls and Target.xls have a value of 303 in cell C2.

And of course Target.xls changes as well as the link is live between the two workbooks. All is as it should be, right?

Step 6 – Close both workbooks

By closing both Source.xls and Target.xls I now have Excel with absolutely nothing open.

If I were to think about the values in the workbooks, they would be as follows:

Workbook name	C2 value
Source.xls	303
Target.xls	303
Target_2.xls	762

Step 7 – Open Target_2.xls

When I open Target_2.xls, I see that the old value of 762 is still in C2. And that an automatic update of links has been disabled. Thank you, Excel 2010, for not exposing me to Bad Things. Alas, not the Bad Thing I am about to (finally) demonstrate.

Step 8 – Open Target.xls

Before I do this, a quick quiz. Yes, it’s Professor Lackpour and I’m here to make your life a living Excel-themed hell. What value do you think will be in Target.xls?

Did you answer 303? Good. That is consistent with what we saw with Target_2.xls – the last linked value (note again that automatic link updates are disabled) displays.

Now the trick question: What value do you think is in Target_2.xls?

Step 9 – The Stupid Excel Trick

Would you believe not the value we saw a moment ago?

Well, you should, because THE VALUE HAS CHANGED.

Let me repeat the steps:

I opened Target_2.xls. Its link-disabled C2 cell value was 762.
I opened Target.xls. Its link-disabled C2 cell value was (and is) 303.
I toggled worksheets (both are in memory, I simply Ctrl-Tab switched to Target_2.xls) back to Target_2.xls. What value did I find in cell C2? Not 762, but 303.
Insert three hours of dumbfounded, flustered, and frustrated experimentation trying to prove that what I saw couldn’t possibly be so.

What I saw, and what you now see, in fact is possible. Not only possible, but documented behavior. And we geeks complain about Oracle not fixing bugs in a timely manner. This one is a whopper and Oracle Corporation, Inc.’s fingerprints are nowhere to be found.

Excel, I HATE you.

Explaining the Stupid Excel Trick

A slight exercise of google-fu brings up this Microsoft support document: Multiple workbooks with links to the same source update each other

At your own discretion, throw in a, “You bastard!” interjection when you read the article; repeat as necessary. Those were at least my exact words.

Let me direct you to the really interesting bit with emphasis added by yr. obdnt. srvnt.:

If you open multiple Excel workbooks that all contain links to the same source workbook, and you choose not to update the links, the links in all of the workbooks will be updated to the values that are stored in the last workbook that was opened.

And this other interesting bit, again with a bit of emphasis:

This behavior occurs because link information is stored in an internal link table. This link table is shared among all open workbooks in that instance of Excel. The most recently opened workbook provides the link table that is in memory at any specific time.

There is a workaround, although not one that anyone who uses the Essbase Excel add-in is likely to use:

To work around to this behavior, open the workbooks in different instances of Excel. As a result, each workbook maintains its own internal link table and stored values.

So just to summarize: Multiple target Excel workbooks that link to the same closed source workbook will reflect the data value in the last opened target workbook.

Some would call this a data integrity problem.

Tim Tow would tell you that’s why Excel is so dangerous. I agree.

Explaining what happened to me, and potentially you

I received an updated data workbook via email. This updated workbook had new values that were linked to multiple source workbooks that I did not have access to.

To be sure (oh, the irony) that I was dealing with the right data set, I opened an older, differently named, version of the workbook.

As both the first and the second workbook were linked to the same closed (and not even available) source files, the first workbook’s linked data values were updated by Excel to the second workbook’s linked data values.

I then sent the data values in the first workbook (which were now wrong because they reflected the values in the second workbook) into Essbase.

Thank you, Microsoft Excel.

Arrrgh. But one must be philosophical about things like this lest they drive you mad. It’s an Excel issue, not an Essbase issue, and not even a Cameron issue (well, mostly).

I don’t think I can say it better than my friend Dave Farnsworth, “Linked workbooks are nothing but trouble and always have been. Shoot anyone who uses them and save yourself the pain. No one who knows Excel will vote to convict.”

Be seeing you.

Motivation

Goethe wrote “You must either conquer and rule or serve and lose, suffer or triumph, be the anvil or the hammer.” Pretty grandiose stuff for an Oracle EPM blog, eh?. Maybe the German saying, “Ambos oder Hammer” which loosely translates to “One must be either anvil or hammer” is more germane.

I quote one of the world’s greatest writers not because this has suddenly become camerons-blog-for-philosophers.com (although that does have a nice ring to it) or camerons-blog-for-developers-that-have-an-overblown-sense-of-imporantance.com (perhaps a bit closer to the truth) but because sometimes a seemingly simple task in Oracle EPM-land (see, I am able to bring things back down to earth) really does pose a Manichaean struggle.

The bad

The seemingly simple problem was this (and yes, this is a bit of an intellectual letdown after the above but bear with me): in a hierarchical securitymodel for Essbase (or Planning or HFM or whatever), how do I know what users have access to an Essbase application?

That doesn’t seem very difficult, does it? As this question was in regard to Essbase, all I need do is simply go to the Technical Reference guide and look for the MaxL statement that lists all of the users that have access to a particular application.

Here’s one in MaxL that’s pretty close: display user in group all ;

That really is pretty close, showing me all of the groups, all of their members, and even the users (I only have two Native users: User1 and User2) in the output.

But actually, that’s more than I want. What if I just want to know what’s in the group ESB_Essbase? Or just the group ES_Sample_Basic? Remember, it’s easy to parse the above as there’s so little output. But what happens when I try to read a production system? Pain, and plenty of it, would be my guess.

So obviously I simply need to change that query so that I only go after ESB_Essbase, right? Wrong.

Now I can only view the children of ESB_Essbase. What about the members that are in ESB_Sample_Basic, User1 and User2? Bummer. What to do?

Also, do you see what none of these MaxL statements do? They don’t:

Give me the full parent/child relationship for a given anchor group (already noted).
Give me security as directly assigned to the groups or the users.
Give me security as implicitly assigned to the users (or the groups).
Give me any of this information in nicely formatted output.

It’s enough to make a geek wail in frustration.

Shared Services to the rescue?

Wait, you say (You do say this, don’t you? I hope you do or else why are you here?), what about Shared Services? Isn’t that where security is assigned to Essbase? And surely if we can assign security, we can report on security? Yes?

No, or at least the security that Shared Services returns is either too atomic or just too much.

Right down to the electrons and protons

I can view the security of one of my users, but only one at a time.

For this one user, this effective roles report is a good start.

But it isn’t enough and I have to know beforehand that Billy Bob (Imagine this former Philadelphia student’s agony when he found out that Billy Bob’s is CLOSED– where oh where will I go for chicken cheesesteaks?) is a descendant of ESB_Essbase. Remember, the whole point was to point at group and get everything associated with it.

The universe

And if I want everything I can right click on Groups in Shared Services and get…

…too much. Again.

Now I really am annoyed. Surely finding out the descendants and their explicit and implicit security for a group in Shared Services (see, I am not just limiting this to Essbase although that is where my quest began) isn’t a big deal? Well, actually, yes, it is a big deal, or at least not something that the Oracle EPM stack provides out of the box.

But as Rabbie Burns wrote, “Ah, but a man’s grasp must exceed his reach.” My grasp exceeds my reach all the time. In this instance, can we resolve the above unanswered questions somehow?

The good

A short introduction to the good

SQL and the Oracle EPM repositories provide the answer. And why not? If security is assigned in Shared Services and that security is stored in the Shared Services relational repository then it is a logical place to look for that security.

That’s the theory, but the actual practice has shown that hacking Shared Services repositories is quite a bit more involved than I anticipated. Given that there’s so much ground to cover I simply am not going to be able to cover this all in one post.

Here’s how I am going to break this up across multiple posts:

The part you are reading right now will be where I illustrate how to get a parent child/recursive table view of an anchor Shared Services group and all of its descendants from the Shared Services repository. This answers the initial question I had long (it seems like a long time ago) ago: what are the users in this upper level group.
The second part will cover what is the direct access of these groups. This is harder than it sounds and it will cover undocumented views in the Shared Services repository.
The last bit, and the one that has given me a lot of trouble, is the implicit and explicit security (given the techniques in number two that actually is no big deal) and even more importantly, how to present that security. I’ll even show two different ways to get there (one by yr. obdnt. srvnt. and a better one by Dan Pressman) and go into some pretty interesting (sort of) recursive theory approaches.

Whew, it’s been a lot of work (I have 99% of the code written at this point and yeah, that last 1% could be a real stinker) thus far.

So where to start?

As always when trying to figure out what’s going on in the schemas, go take a look at the excellent schema overview in the Deployment and Installation section of the 11.1.2.3 documentation.

It’s all there except for the undocumented views (go on, take a look at VW_APPLICATIONS – yes, I will be covering this view in the second post) and you will need (or at least I needed) to spend some quality time pondering what each one of these tables do and how to query them.

But that’s all background. Let’s start out with something easy – a recursive query to get all of the descendants of an upper level Shared Services group. I’ve written about this before within the context of a Planning query and, if you attended my Practical SQL for EPM Practitioners at Kscope13 (believe it or not, you can hear yr. obdnt. srvnt. present this session as it was recorded; of course you can download the presentation from ODTUG’s web site), I covered how this works in pretty good detail.

If you’re not familiar with recursive queries (all three parts of this series will use them), and if you want theory, I really suggest you download the above presentation as I go into how a recursive query works. No worries, I’ll wait for you to read/listen to the presentation. I’m not going anywhere.

Skip the theory, go right to the code

All done? Or you already knew all about recursive queries? Terrific, as we can now dive into the code.

Tables you should know

This whole query only uses three tables to get the groups, members of the groups, and native users. Yes, I cheat a bit with users as I do not have a MSAD/LDAP environment to go after the users. That gets a bit more complicated but can be done.

CSS_GROUPS

Native groups are stored in this table. I have read about, but never seen, MSAD/LDAP groups in Oracle EPM environments. They make sense if your firm’s IT department are moving MSAD/LDAP users in and out of groups but the norm seems to be Native groups.

It’s a bit hard to read, but here are all of the groups in my EPM instance.

Note that I use LOWER_IDENTITY_ID as I never have to worry about case or throwing LOWER or UPPER functions around id codes.

CSS_GROUP_MEMBERS

This table does just what its name suggests: members in groups.

Note that MEMBER_TYPE can be either a 1 or a 2. 1’s are uses, 2’s are groups.

CSS_USERS

As I wrote, this is where I cheat a bit, as I only have Native users on my 11.1.2.3 VM. You will likely need to go against CSS_IDENTITY which stores external users and have some sort of way of getting at your directory services’ user names.

In any case, here’s what my CSS_USERS table looks like and only a bit of it because there are so many fields. Again, I suggest you read the docs if you want to look at all of the fields.

The code

Here it is in SQL Server’s T-SQL.

-- NB – My VM’s Shared Services repository is in a SQL Server

-- database called, unsurprisingly, HYP_SharedServices

USE HYP_SharedServices

WITH TopCat AS

(

-- Set up the anchor member

SELECT

G.LOWER_IDENTITY_ID AS'Parent_ID'

, G.LOWER_IDENTITY_ID AS'Child_ID'

, G.NAME AS'Parent'

-- Need to CAST as VARCHAR(200) to allow UNION ALL

,CAST(''ASVARCHAR(200))AS'Child'

, 1 AS'Gen'

FROM CSS_GROUPS G

-- This is where you put in the topmost member of the tree

WHERE G.NAME ='ESB_Essbase'

Related AS

(

-- Almost Parent/Child, but not quite

-- I made this a separate CTE because it's cleaner down in the true Parent/Child query

SELECT

GM.LOWER_GROUP_IDENTITY AS'Parent_ID'

, GM.LOWER_MEMBER_IDENTITY AS'Child_ID'

, G1.NAME AS'Parent'

-- 1 = user, 2 = group; Cast both as VARCHAR(200) to allow UNION ALL

,CASE

-- Subqueries for Users and Groups to get the names

WHEN

GM.MEMBER_TYPE = 1

THEN

(SELECT

CAST(U.NAME ASVARCHAR(200))

FROM CSS_USERS U

WHERE U.LOWER_IDENTITY_ID = GM.LOWER_MEMBER_IDENTITY)

WHEN

GM.MEMBER_TYPE = 2

THEN

(SELECT

CAST(G2.NAME ASVARCHAR(200))

FROM CSS_GROUPS G2

WHERE G2.LOWER_IDENTITY_ID = GM.LOWER_MEMBER_IDENTITY)

END

AS'Child'

FROM CSS_GROUP_MEMBERS GM

INNERJOIN CSS_GROUPS G1 ON

GM.LOWER_GROUP_IDENTITY = G1.LOWER_IDENTITY_ID

-- Bring it all together to figure out the recursive table output

ParentChild AS

(

-- Set the anchor

SELECT

T.*

FROM TopCat T

UNIONALL

-- Oooh, fancy, we're going recursive here

SELECT

R.Parent_ID

, R.Child_ID

, R.Parent

, R.Child

, PC.Gen + 1

FROM Related R

-- This is the recursive JOIN that ties child to parent

INNERJOIN ParentChild PC

ON R.Parent_ID = PC.Child_ID

)

SELECT*FROM ParentChild

The results

And all of that leads to this single group list of descendant Shared Services users and groups:

Btw,Parent_ID and Child_ID are not needed for the purposes of reporting this information but I use them later to get security so they are sticking around. Comment them out in your own environment.

Were I to change the above query’s anchor member from ESB_Essbase to PLN_SampApp1, I would get the following:

That seems like an awful lot of work to get a simple list of an upper level group and its descendants but it all becomes worthwhile later in this series. I like to think, in my humble way, that I dive into some really interesting SQL techniques (that is half the idea of these query series) thanks to the Great False God Google (from whom I found and stole techniques and some intersting theory) and, as I mentioned before, Dan Pressman who showed me a better way to create good looking security reports.

What do we have?

Well, has Oracle’s EPM system been hammer or anvil? This blog post lends credence to the argument that good has triumphed over evil, or at least over annoyance, and that I have made the EPM Shared Services schema bend to my will. So I suppose one could argue that I was the hammer.

On the other hand, as this took an embarrassingly long time to figure out, I have to sort of wonder if maybe I more resemble that anvil. My head does kind of hurt and something has been hammering away at it. Oh well, at least I got to write some interesting code and learn new stuff.

What’s next?

A parent child table is great, but I want more, more, more.

How about the above but tied to directly assigned:

Roles
Projects
Applications
Products

Fwiw, yes, yes, I know that you can export out Shared Services to LCM and then look at the .csv files that result (I did this earlier this year because I was too dumb/pressed for time to write the above queries/they wouldn’t give me SELECT access), but my query is faster, easier, and more customizable as well.

Neither native Shared Services nor LCM will report implicit security for all users in a group, by level of hierarchy. A SQL query will as I will show in part three of this series and that is the power of SQL and why every EPM practitioner should cultivate these skills.

Be seeing you.

Provisioning

As I wrote in my first post on this subject, after a bit of pain around figuring out how to read and understand the Shared Services schema, I can now run a query that will give me all the descendants of a given Shared Services group. Huzzah for me! Ahem.

But while that is nice (and the pat on the back I just gave myself felt oh so good), I want, no I need to know what the provisioning is for all of those descendants of a Shared Services group.

Are any of us, really, immune from the seven deadly sins? I can’t imagine how I would be, and certainly greed (along with the rest) is one of my faults. Wait, is this camerons-blog-for-theologians.com? Or camerons-blog-for-wearing-a-hair-shirt.com? Or camerons-blog-for-repentance.com?

Nope, the subject is Essbase, not original sin, and you’re reading camerons-blog-for-essbase-hackers.com. But I’m not taking back my desire to wring more out of Shared Services than I originally claimed I wanted. As Gordon Gecko said, “Greed is good”. Or maybe I should call this blog the outcome of Adam Smith’s Invisible Hand. Although I am not totally clear on how I maximize my economic gains by writing this beast.

Back to a semblance of relevance – can I marry up the parent child list to directly assigned security? Can I? Hopefully, or I won’t have much of a blog post.

Roles

Given that Oracle EPM roles are defined in Shared Services, it doesn’t seem shocking that the table CSS_ROLES contains, well, the Oracle EPM roles. But did you know there are 191 of them?

Here’s a simple query (it will become part of a Common Table Expression a bit later on) that combines CSS_ROLE with CSS_ROLE_LOCALES (which stores localized by language role names and description).

-- Figure out all of the possible roles in each project

SELECT

R.LOWER_IDENTITY_ID

, R.ROLE_ID

, R.PRODUCTCODE

, R.PRE_CONFIG_STATUS

, L.VERSION

, L.NAME

, L.DESCRIPTION

FROM CSS_ROLES R

INNERJOIN CSS_ROLE_LOCALES L

ON R.LOWER_IDENTITY_ID = L.LOWER_IDENTITY_ID

WHERE

LOWER(L.LOCALE)IN(LOWER('en_US'))

AND L.LOWER_NAME LIKELOWER(N'%')

ANDLOWER(L.VERSION)=LOWER(N'11.1.2.0')

Which gives us something like this:

By the way, if you wonder how I came up with that WHERE statement I am not ashamed to say that I used SQL Server’s Profiler to watch (aka, steal) SQL code as it came out of Shared Services.

I can almost hear you say, “Wait a minute. Are you saying that you set up SQL Server Profiler on your VM and then just regurgitate the queries back to us, while claiming that you wrote it all yourself?” I so wish that were the case as the time it would take to write these posts would be quite a bit quicker. Alas, no.

The query you see above is the only bit of code (okay, there is one more snippet) I was able to “borrow” from Oracle. What does Shared Services (and all of the other Oracle EPM tools – trust me, I’ve looked) do? Take a look at the lines that say “exec sp_execute 1” and “declare @p1 int set…”. I am here to tell you some of what the products do are “normal” queries and much of the rest is done within the application, including the population of those stored procedures which btw destroy themselves on execution. This overall query, other than this one bit (again, with one other qualification) is, for better or worse, 100% Cameron. Probably worse, but I will persevere on.

Applications

But looking at SQL Server Profiler showed me something else: use of a view called VW_APPLICATIONS. A quick perusal of the Shared Services schema guide shows exactly no mention of that view. What oh what oh what could that view provide?

Code

Go take a look at the view if you wish. Wait, this is the second code bit that is Not By Cameron, but again, I am pointing this out, so it isn’t exactly subterfuge or fraud on my part.

NB – If you stick the above code into a query, make sure you remove the last parentheses.

Fields

Output

Oooh, isn’t that pretty? We have nothing less than a nice view of all of the applications in pretty almost-English. Click here to embiggen the picture to see all of the apps on my VM.

3 + 1 +1 = 5

Before I confuse myself with higher mathematics, it’s really quite simple.

3 = the CTEs TopCat, Related, and ParentChild that you saw in our last thrilling installment
1 = the CTE RolesByProduct which I illustrated above
1 also = the CTE Direct Security of which more anon

Put them all together and the answer isn’t really 5 but a direct (as in what is directly provisioned to the inclusive descendants of a Shared Services group) report on group hierarchy and the security assigned at each level.

The code

USE HYP_SharedServices

WITH TopCat AS

(

-- Set up the anchor member

SELECT

G.LOWER_IDENTITY_ID AS'Parent_ID'

, G.LOWER_IDENTITY_ID AS'Child_ID'

-- Need to CAST as VARCHAR(200) to allow UNION ALL

,CAST(G.NAME ASVARCHAR(200))AS'Parent'

,CAST(''ASVARCHAR(200))AS'Child'

, 1 AS'Gen'

FROM CSS_GROUPS G

-- This is where you put in the topmost member of the tree

--WHERE G.NAME = 'PLN_SampApp1'

WHERE G.NAME ='ESB_Essbase'

Related AS

(

-- Almost Parent/Child, but not quite

-- I made this a separate CTE because it's cleaner down in the true Parent/Child query

SELECT

GM.LOWER_GROUP_IDENTITY AS'Parent_ID'

, GM.LOWER_MEMBER_IDENTITY AS'Child_ID'

, G1.NAME AS'Parent'

-- 1 = user, 2 = group; Cast both as VARCHAR(200) to allow UNION ALL

,CASE

-- Subqueries for Users and Groups to get the names

WHEN

GM.MEMBER_TYPE = 1

THEN

(SELECT

CAST(U.NAME ASVARCHAR(200))

FROM CSS_USERS U

WHERE U.LOWER_IDENTITY_ID = GM.LOWER_MEMBER_IDENTITY)

WHEN

GM.MEMBER_TYPE = 2

THEN

(SELECT

CAST(G2.NAME ASVARCHAR(200))

FROM CSS_GROUPS G2

WHERE G2.LOWER_IDENTITY_ID = GM.LOWER_MEMBER_IDENTITY)

END

AS'Child'

FROM CSS_GROUP_MEMBERS GM

INNERJOIN CSS_GROUPS G1 ON

GM.LOWER_GROUP_IDENTITY = G1.LOWER_IDENTITY_ID

ParentChild AS

(

-- Set the anchor

SELECT

T.*

FROM TopCat T

UNIONALL

-- Oooh, fancy, we're going recursive here

SELECT

R.Parent_ID

, R.Child_ID

, R.Parent

, R.Child

, PC.Gen + 1

FROM Related R

-- This is the recursive JOIN that ties child to parent

INNERJOIN ParentChild PC

ON R.Parent_ID = PC.Child_ID

-- Figure out all of the possible roles in each project

RolesByProduct AS

(

SELECT

R.LOWER_IDENTITY_ID

, R.ROLE_ID

, R.PRODUCTCODE

, R.PRE_CONFIG_STATUS

, L.VERSION

, L.NAME

, L.DESCRIPTION

FROM CSS_ROLES R

INNERJOIN CSS_ROLE_LOCALES L

ON R.LOWER_IDENTITY_ID = L.LOWER_IDENTITY_ID

WHERE

LOWER(L.LOCALE)IN(LOWER('en_US'))

AND L.LOWER_NAME LIKELOWER(N'%')

ANDLOWER(L.VERSION)=LOWER(N'11.1.2.0')

-- Figure out direct security assignments for all levels of the hieararchy.

-- Use the undocumented view VW_APPLICATIONS to get a pretty descriptions of the

-- products and projects.

DirectSecurity AS

(

SELECT

--PC.*

--, I.LOWER_ROLE_IDENTITY

--, I.LOWER_APPLICATION_ID

PC.Parent

, PC.Child

, PC.Gen

, VA.PROJECT_NAME

, VA.APP_DISPLAY_NAME

, R.NAME

, R.DESCRIPTION

, VA.PRODUCT_NAME

FROM ParentChild PC

LEFTOUTERJOIN CSS_PROVISIONING_INFO I

ON PC.Child_ID = I.LOWER_MEMBER_IDENTITY

LEFTOUTERJOIN VW_APPLICATIONS VA

ON I.LOWER_APPLICATION_ID =LOWER(VA.APPLICATION_ID)

LEFTOUTERJOIN RolesByProduct R

ON I.LOWER_ROLE_IDENTITY = R.LOWER_IDENTITY_ID )

SELECT

DS.*

FROM DirectSecurity DS

Here’s the Essbase-related group ESB_Essbase.

And here’s the Planning-related group PLN_SampApp1. And yes, I really did assign Calculation Manager rights to a lower level group just to see what the query did with it.

Inner versus Outer

No, I am not talking about belly buttons but instead the somewhat unusual use of LEFT OUTER JOINS at the end of the DirectSecurity CTE. These are necessary because the users (you are looking at a proper hierarchical security model after all) have no security. Trying to INNER JOIN these users across provisioning, applications, and roles would result in them being dropped. Instead, LEFT OUTER JOINs are used to include them in the output. Go on, play with the code and you’ll see what I mean.

What’s next?

While it’s nice to have a report that shows the hierarchical relationship between groups, sub-groups, and users along with their security, this is a top down report.

What about a query that gives me all of users in the application, their implicit and explicit security, and the hierarchical group lineage so you can see how that provisioning gets assigned? Yes, that is available on a user by user basis in Shared Services but not as a group report. Now I just have to finish that bit of code…

Be seeing you.

This blog in review

I’ve been writing, working, and in general sweating over this blog for almost five years. That is sort of a shocking statistic, but I started in May 2009 and here we are at the beginning of 2014. Okay, so sue me, four and a half years, not five. It’s true, I promise you. (You have to watch the link to get the sort-of joke.) Fwiw, I can’t decide if Frank Sinatra (Nathan Detroit) or Vivian Blaine (Adelaide) is the better in this scene.

That is a sobering amount of time and effort to apply to one project, especially an unpaid one. Why do it? Indeed why?

At the risk of sounding maudlin, I do it for three reasons, in order of decreasing selfishness:

Putting pen to paper (okay, keyboard to editor) forces me to really think out ideas. Oft times really thinking out ideas means, “I thought this could be done; now I have to teach myself how to do it.” I write for this blog, but ultimately I learn from the writing.
In my own (exceedingly) small way, what I write moves the collective knowledge of EPM practitioners just that little bit forward. I always know that I stand on the shoulders of giants (so perhaps this blog should be entitled Cedalion’s Blog for Essbase Hackers) but I do my l bit.
I know that you, Gentle Reader, get value from this blog. How? Emails, people I meet at conferences (The requests for photographs are weird, but not as surreal as the guy that insisted that I was “awesome” at UKOUG 2010. I like being classified as awesome but I have not been to Dear Old Blighty since 1996 so someone was robbed of the awesomeness that was rightfully his. Of course everyone is welcome to talk to me at conferences, assuming I am not running desperately towards a meeting I am five minutes late for, which is most of them.), and the statistics that this blog generates.

And that, really, is what this blog post is about – you: where you come from, how you get here, and what you care about. I am no mind reader, but instead a devotee of Google Analytics (GA). Surely all EPM devotees are interested in analytics. This bit of blog-based navel gazing is more than mere indulgence but is instead a view into what the EPM world cares about, circa 2013.

A bit of sobering information

You’re about to see lots of numbers, and think, “Wow, Cameron sure is full of himself, bragging about how popular his work is.” Lest you think this, know that once upon a not too distant ODTUG board meeting, we bloggers were asked to reveal how popular our blogs were. I was quite chuffed with my ~3,500/month visits. I was however, mildly shattered when Tim Tow revealed that he got more than 17,000 per month. Maybe Tim should do a post like this. :) I have no insight into other blogs like John Goodwin’s, or Glenn Schwartzberg’s, or Edward Roske’s, or whoever’s, but I imagine that their numbers dwarf mine.

Regardless, this is Cameron’s, Not Someone Else’s Blog For Essbase Hackers, so on with your numbers, courtesy of GA.

How many are you?

2013, A.D.

In 2013, there were just under 23,000 of you. You made over 40,000 visits. Thanks to Blogspot’s (owned by Google) cookies on your computer (please do not blame me for this – it’s the way it works and I have no control over this), GA can track how often you visit.

Year over year

Depending on how you want to look at it, one could argue the blog is up about a third in visits, visitors, and page views.

From the beginning

How many of you have read this blog from its inception? Just under 60,000, according to GA. You can see how the audience growth really hit its stride in 2012.

For giggles, I like to look at BlogSpot’s own numbers which show “better” values. Alas, the difference in data is due to all of the automated web crawlers that search for content. GA is able to filter that out, hence the lower numbers.

Regardless, there’s a lot of you reading this blog, for which I am thankful.

Where are you from?

All over the world, actually. Not all that surprisingly, the United States is my primary audience. India, given its population and IT presence, is also not a shock. I am pleased that my (distant) cousins in the United Kingdom are number three and am amazed at Australia’s high rankings, although I have to put that down to my ODTUG Seriously Practical conference.

YoY by country

You can see that although the absolute numbers in the US are higher, the blog is becoming more international in its readership as the States are actualy just over 2% down as a percentage of total readers.

Otherwise, in the top 10 countries, it’s a case of growth, except for Italy. I put that down to the economic disaster that is southern Europe.

What about the U.S. of A.?

What’s up with Wyoming? I guess cowboys don’t need Essbase.

Want to know where the most Essbase activity exists in the States? Take a look at the top ten.

To be absolutely fair, I am not sure that the top ten stats really and truly are where the most Essbase activity exists as I seem to have consulted in most of those states at one time or another. It could be a trailing effect (assuming that I do not leave my clients in a state of outrage) from me being there.

Also, it isn’t surprising that large populous states have a lot of readers but it is still interesting to see how the numbers fall out.

YoY by state

This blog was super popular in Tennessee, Florida, Georgia, and New Jersey. The other top ten states saw growth, but not to the same level. California was almost flat – hmm, what do I have to do to appeal to them?

Who are you?

No, I can’t tell who you are, really, but I can tell (in aggregate) where you are from and your primary language.

Your providers

This is primarily a US blog, so it is not a shock to see Comcast trump Verizon.

Individual corporations are harder to spot as so often they use the big providers (including the intriguingly named “Internet Service Provider” one). It is kind of cool to see that Oracle reads this blog, if only to identify the mistakes and have a good laugh.

Tower of Babel

English (‘Murican and Queen’s Own), French, Spanish, Russian, German, and Portuguese (Brazilian) are the primary languages.

How did you get here?

Most of you came from Google, then bookmarks, then Bing (I am having visions of Bing Crosby doing one of his jazzier routines), and then blogs.

I find it interesting that Edward’s blog is ranked number five as it is (mostly) not updated very often. Of course, when there is a new post, an awful lot of people read it, hence the high referral rate. Or maybe they go there first, get disappointed at the lack of new content (Edward, can you tell I am beseeching you to write more?), and then in reluctance turn to my blog.

What do you use to read this blog?

The top ten are no great surprise. Explorer continues as the corporate standard -- people who don’t work for corporations don’t much care for Essbase, as hard as that may be to believe. As a Firefox fan, I am surprised at Chrome’s second place finish. I love the many add-ons that Firefox allows and as far as I can tell, Chrome is a take-it-or-leave-it proposition. To each his own, I suppose.

Not much of the readership is on mobile if I understand how browsers get reported. Android devices pip Apple’s products, but just by a hair.

What do you care about?

Ah, here’s the interesting bit. Other than the first entry, which represents this blog’s home page, I can tell that in 2013 you cared about:

Data export
Calculation Manager
Hyperion Business Rules
BSO Backups
Making Planning’s filters METAREAD
11.1.2.1 (this one puzzles me given the latest release is 11.1.2.3)
Error handling in MaxL
Offerings from Oracle Support
Various Essbase tricks, features, and frustrations
Data export approaches from Essbase
Amazon Web Services

If you look at the year of the post, you’ll see that of the top 20, only one post, the on an undocumented MDX function called NONEMPTYBLOCK, came from 2013. The rest of the entries are “famous” in that they answer perennial questions and thus get read a lot. Note to self -- you must find more undocumented MDX keywords and blab about them to world+dog.

Where do we go from here?

The above is sort of like hitting you over the head with a spreadsheet. Numbers yes, but what does it all mean?

I think this blog’s continued success, as borne out by the above data, mean a couple of things:

A slightly changed emphasis in effort
Deeper, longer, blog posts
Proof that I am doing the right thing

A slight change of emphasis

I do spend quite a bit thinking about what I write and even more time doing the writing. I am very encouraged by your support of this blog and am committed to continuing its success. In an attempt to have a life/be gainfully employed/do all of the other volunteer work I have purposely cut back a bit from the forums as I simply can’t do everything I want to do and do it at the same (at least to me) high level.

Don’t worry (or maybe rapture is the correct emotion if you are not a fan of my forum contributions), I will not be withdrawing from Network54/OTN by any means, but my day is 24 hours, just like yours, and I can’t create time where none exists.

More of the same, but better

With that in mind, what I can promise (I hope) is more of the deeply researched posts with the same attention to detail. Yes, sometimes the posts will be slightly shorter (I have one on Excel coming up that isn’t very involved although it lost me an afternoon of my life trying to figure it out) and yes, the same linked references to things Not Oracle EPM will continue. As always, don’t bother clicking if you don’t wish to, but I do hope that you too have interests outside of work, and that maybe you’ll share some of my admittedly odd interests.

Hopefully, this is the right thing to do

As Maurice Chevalier once sang in I’m Glad I’m Not Young Anymore, “I’ve never been so comfortable before”. What do I mean by this? No, I will not be starring in a Broadway revival of Gigi (the mind recoils). Neither is Methuselah my patron saint. But consistently writing this blog for so long has given me the perspective and confidence of hard data. Experience does sometimes equal serenity and if I will never have the je ne sais quoi of Chevalier then at least I have the numbers as noted in the above sections that prove what I’m writing is useful to more than just me.

Maybe a slightly less cynical take on all of this is by Messrs. Armstrong, Crosby, Sinatra, and the exquisite Miss Lee in their opening for the 1959 Bing Crosby Oldsmobile Show (What, you can’t see me as a fan of Der Bingle? It’s true, I promise you (to quote a song), as I have pretty much rejected all of American culture post-1965.). This blog has been around, and perhaps its forevermore is shorter than before, but I’m glad it’s not young anymore because the best is yet to come. Oh, the puns, they hurt.

It’s all about you

Something that we in the EPM world might forget is that numbers are meaningless without context, and in the case of this blog, that context is you. I could write until the cows come home but if you didn’t read it, there wouldn’t be much point.

Be seeing you.

Is time travel possible?

Alas and alack, no.

Although I have searched for just such a device, to my knowledge there is no WayBack Machine that can take me back to 1992. Yes, that long distant time from the 20th century before the Global War on Terror, before the Internet (at the time I had an AppleLink account which gave me email to someone actually outside of my Fortune 50 employer as email used to stop at the company network’s end – it was a different time), a time when I did OLAP on an IBM mainframe using Comshare’s System W, and a time when all of the hair on my head was 100% devoid of grey.

22 years is a fair clip ago when it comes to personal history and it is an eternity in the technology field.

And yet there has been in my professional life a constant since that time (for me it started in 1993 or 1994 but give me artistic license).

What oh what oh what could it be?

Note to young people (like, oh, under the age of 30) – once upon a time, people (honestly, even the women had to wear ties of a sort) in corporate America were expected to wear a suit and tie (all the time – to the bathroom, to lunch in the cafeteria, on the way out the door – always), took notes on paper (actually, I still do), had 3270 terminals at their desks (nope, don’t have one of those, thankfully), etc. In other words, “The past is a foreign country: they do things differently there.”

With that preamble, what might be the one constant from early in the Clinton administration?

No, not suits – those are gone except for weddings and funerals.

No, not ties – see the above.

No, not rolodexes – they were obsoleted by smart phones.

No, not mainframes – they are certainly around but are no longer the primary target of corporate IT.

It’s Essbase. Yes, Essbase is that old – over 22 years. If Essbase were human, it could vote, marry, take a loan out for a house, buy a gun, and serve in the armed services. In other words, Essbase is a mature product.

That it is still around, largely in its original form is testament to its genius. Yes, there have been many improvements and additions, but at its core, Essbase still does what Essbase did when it was first invented by Bob Earle and Jim Dorrian back in 1992. Don’t believe me? Take a look at the patent for BSO Essbase. Yes, of course ASO Essbase is internally quite different, but read that application and see if what was described is all that functionally different than what it does today.

Not a time machine, but something almost as good

As I wrote, I haven’t found a time machine. But I recently did get to spend a fascinating afternoon with one of the two fathers of Essbase – Bob Earle. I must hasten to add that this is because of the generosity and contacts of my good buddy Tim Tow who seemingly knows everyone.

Fellow ODTUG board members Tim, Natalie Delemar, and I met Bob for a few drinks at a hotel in Seattle. We were all there because of the ODTUG board face to face meeting (we have two a year, one right before Kscope14, one six months earlier, usually in the host city). This was our chance to meet one of the progenitors of Essbase and we were all quite keen to meet him.

Here we are.

Natalie, Yr. Obdnt. Srvnt., Bob Earle, and Tim Tow

So what was it like?

Bob is an incredibly self-effacing guy. He was astonished that Essbase was still around, amazed that people cared about it so much, and was, I think, sort of gobsmacked that we were so interested in his story. He was also very clear that he was only one half of the team that invented Essbase.

Essbase was an idea, an opportunity, and (I think) a really fascinating and stressful chapter in Bob’s life.

I will also note that Bob is a very private person, and asked that what we discussed remains confidential. I will say that I think he found the genesis of Essbase to be quite the roller coaster and he has completely divorced himself from the high technology world. Take from that what you will.

It was a true honor to meet the man that set so many of our careers on the path they now are. I’ve often thought of what I might be doing if my former boss, Mike Rose, hadn’t thrown Comshare’s resold version of Essbase on my desk and said (I am paraphrasing, but this is pretty much as it happened), “My mainframe costs are killing me, see what you can do with this product.”

The rest, as the saying goes, is history.

Here I am with the guy that unwittingly landed me right here at this blog.

Sometimes you can go home again.

There is more to BI/EPM conference life than Kscope. I know, hard to believe, but it’s true, I promise you. Don’t believe me? Read on, Gentle Reader, and all will be revealed.

Sometimes a chance remark makes things happen

Over on the Network54 Essbase page, there was a thread about regional user conferences in the days of Hyperion and how there really isn’t anything today to take their place. Never one to pass up stealing other people’s ideas miss an inspiring thought, I took this comment from the mysterious “RIM” to ODTUG, ODTUG did their usual hard work, and the result of their effort is this regional Chicago ramp-up to Kscope14.

The speakers below are all Kscope veterans – think of this as a preview to Kscope and all the more reason why you should be in Seattle as well.

When, where, what, and how to sign up

When

21 February 2014, 12:45 to 5 pm

Where

Oracle’s regional Chicago office, 233 South Wacker Drive, 45th floor, rooms 45014/45015/45016

What

Shared Services API – Oracle ACE Celvin Kattookaran
Exalytics Performance – Oracle ACE John Booth and Tim German
BI Solutions with Oracle Analytics – Dave Collins
Financial Close Management – Jim Springer

These are fantastic speakers and content and the cost is absolutely zero, zilch, nada, you get the idea – FREE. It is rather difficult to argue with that price.

How

If you are interested, and I cannot imagine how any Chicago area BI/EPM practitioner would not be, sign up here.

Thanks to all of the speakers for taking time to do this, ODTUG for organizing it, and of course Oracle for providing the meeting venue.

I wish, wish, wish I could be at the meeting but alas I cannot. It will be that good and it is a regional user conference not to be missed.

What’s Round on the Ends and High in the Middle?

O-hi-o.

Sorry, that joke has been bouncing around my head since I saw the film Third Finger, Left Hand and heard Melvyn Douglas sing that particular ditty. Lest you roll your eyes at yet another old movie reference, take a good look at Myrna Loy. Phwoar. It's also a pretty funny movie.

Why the reference to the Buckeye State? That’s where the Northeast Ohio Oracle Users Group (NEOOUG) is holding their annual Great Lakes Oracle Conference (GLOC). If you are an Oracle practitioner in the Midwest, this conference is right up your alley, as is the user group itself.

When, where, what, and how to sign up

When

The Great Lakes Oracle User Conference is 13 to 14 May, 2014. The pre-conference seminars (of which more anon) are 12 May.

Where

Student Center 3rd floor ballroom foyer at Cleveland State University

See here for hotel and more detailed information.

What

GLOC is not primarily a BI/EPM user conference. Yes, for Essbase hackers aka the Best and Brightest aka whoever reads this blog, it is sometimes difficult to realize how small a part of Oracle’s universe we are but it is The Awful Truth. Don’t view that as a bad thing, but instead see it as an opportunity to broaden your horizons. You may have noted that this blog actually covers a whole bunch of technologies, Essbase included. I do that not because I am sort of wonderful renaissance man, but because that is simply what it takes to succeed in IT. I expect you do the same.

With that broad technology interest in mind, take a look at what NEOOUG have on offer.

Keynote speakers

GLOC has not one but two keynote speakers:

Tom Kyte – Sharpening your Memory: A look at the Oracle in Memory Database
Steven Feuerstein – Coding Therapy for Software Developers aka How Does This Code Make You Feel?

I saw Tom present at NZOUG 2013 and he was not half bad. :) All kidding aside, Tom is a bit legendary as is Steve. Both sessions will be beyond interesting.

Session Tracks

As I wrote, this conference (just like Kscope) is all things to all men, or at least tries very hard to speak to much of the world that defines Oracle:

Application Development
DBA
Data Warehousing/BI (see, I lied, there is BI/EPM content here)
System Architecture/Administration

That’s 30 sessions in total over two days and these look like really geeky sessions based on their titles. Click here to Read The Whole Thing.

Still accepting abstracts

Despite the above agenda, the deadline for abstracts is still open, and will be till 16 February 2014.

Workshops

GLOC kicks off with a full day of preconference workshops.

So far, the workshops consist of:

Oracle Performance Tuning 101 – Carlos Sierra and Mauro Pagano
From Relational to Hadoop – Migrating Your Data Pipeline – Alex Gorbachev
APEX Hands-On Crash Course – Scott Spendolini

Speaking of which…

GLOC’s workshops will also have (assuming of course that NEOOUG accepts my summary and abstract, so maybe yes, maybe no) a three hour Introduction to Essbase seminar given by yr. obdnt. srvnt. to be given on 12 May 2014.

You say that you don’t see the workshop on the link (at least at the time of the writing of this post)? That is because I have (ahem) yet to submit the summary and abstract. All I can plead is that I am a tad over committed. Sometimes I even get to sleep – really, if I could just remove that trifle of biological function, I’d be set from a do-all-the-cool-stuff-I-do-outside-of-work. Thomas Edison claimed that sleep was a waste of time. A man can dream. Or not.

Putting aside unrealistic dreams (ahem yet again), I did just suffer a two day power outage complete with downed power lines trapping me on my street. I viewed this as a gift as it allowed me time away from an awful lot of distractions (you know, like paying work) and I have now written, on paper (hey, no power = no typing it up) a seven page outline of what I want to cover. Ambitious? Yes, but I think it will be a killer intro to Essbase.

Here’s the summary

To relational people, Essbase is a strange creature. It’s not a relational database, but it’s owned by Oracle. Essbase can speak SQL, but you cannot run a SQL query against it. Financial users love it, IT departments are puzzled by it. What is Essbase and why should you care? This three hour seminar will give you a firm grounding on what Essbase is, how it works, and what it’s like to develop in Essbase.

I've got fingers crossed on the abstract getting accepted.

How

Signing up couldn’t be easier. Simply click right here and sign up. You’ll (hopefully) soon see my workshop there. I’ll be submitting the abstract just after this blog post goes up, I promise you.

Wrapping it up

So yes, there is quite a bit more to user group life beyond Kscope.

The first regional Kickoff to Kscope user group meeting (may it be one of many that ODTUG will put on in future) is in Chicago and is focused on BI/EPM. If you are in the area, it is a do-not-miss event. It’s coming up soon, 21 February 2104, and you can sign up right here.

The second, rather larger, Great Lakes Oracle Conference is also chock full of awesomeness. Maybe it will even include me and my ever so slightly awkward love of all things Essbase. Don’t forget that it isn’t too late for you to submit your own abstract for a session and have the conference fee waived.

Be seeing you.

How do I know when I’ve spent too much time in Excel?

When I see this:

That’s a Classic Essbase Excel add-in message and proof that I was doing waaaaay too much analysis of a particularly knotty data issue. In almost 20 years of Essbase work (most of it with the add-in) I have never managed to get that error message. I took it as a note that it was time for me to stop and take a coffee break.

But that’s not what this blog post is about. What this blog post is about Yet Another Stupid Excel Trick because of Yet Another Manual Process.

I must give mention of Mabel Van Stone who tried to warn me of this issue. As usual, I heard it, but didn’t hear it. Now I am going to write it. Maybe this time it will actually sink in.

The hell that is linked workbooks

If you read my last blog post on this subject, you’ll know that I am not exactly a terrific fan of linked Excel workbooks because of the potential of completely mucking up the data.

I also am not exactly a terrific fan of: mean people, stupid people (although on reading this post you may be convinced that I am part of that group), injustice, bad coffee, and a whole host of other things. Me not liking them matters not a jot as they exist with or without my approval. What does matter is knowing what defines “bad” and then firmly following one of two paths: avoidance of things I cannot change (and grudging acceptance therof) and the complete eradication and expiration of the things I can change. Sometimes run and hide is the answer, other times violence is the answer. Yes, I am a simple man. What I am about to illustrate falls into the former category.

Source and linked target open at the same time

Let’s begin with a simple (hah!) example of linked workbooks, this time with both workbooks open.

Source

Do the products in column A look familiar? No? They are Cameron’s detail skus to MVFEDITWW (pronounced mmmfeditwwwuuua) aka Sample.Basic.

Target

And here is that linked target in MVEDITWW’s send workbook:

Note that the total Excel formula value in the send sheet’s row 6 is linked to the target sheet’s row 4.

Updates

A change in the source:

Is reflected in the target immediately as both workbooks are in memory:

Sheet renames

If the source sheet is renamed from “Source sheet” to “Source sheet 1”:

It is reflected immediately in the target workbook formula:

All very slick and goof proof. Maybe.

Closing the files

That’s great if you have both workbooks open, but it’s often common practice, aka some form of sanity to keep just one workbook open at a single time so what’s being dealt with is obvious.

To keep an even greater degree of sanity, it is also common to disable the automatic updates of linked workbooks. Maybe you want the target workbook to be updated, maybe you don’t. The only way to control this is to:

Go to the Data ribbon
Click on Edit Links
Click on Startup Prompt
Click on Don’t dipslay the alert and don’t update automatic links

This then puts you in control of updating or not.

I like this because I in general like control of my data and also because it makes this example easier to demonstrate. NB – the functionality is the same if automatically updated so that is not a free ride to getting away from the Excel Stupid Trick.

Closed source workbook

So what happens when data gets changed in the source and then saved and closed?

It’s not updated until it is explicitly updated by you, oh Excel god.

Updating is easy, simply go to Data->Edit Links->Update Values and the new numbers are reflected in the workbook. Simple and again you are in control.

Renames

What happens if you rename the old sheet and then save and close?

On update, Excel knows that the sheet formerly named “Source sheet” is now called “Source sheet 1”. It is not tied to the number of worksheets in case you were thinking it went off of the index of sheets in a book. Somewhere in the depths of Excel, there’s a code associated with the sheet name. We see it as “Source sheet 1” or whatever; Excel has its own name that allows this kind of renaming whilst retaining the links.

Note that before the update, the sheet name is the old “Source sheet”. After the update it is “Source sheet 1”.

This is all pretty awesome, isn’t it? What oh what oh what could possibly go wrong?

New sheet with the same name

If I create a backup of the old sheet (hey, I may want to go back to the old data) and create a new sheet with new data, what happens?

In the example above, Excel used (I think) some sort of internal sheet code to keep the link to the renamed sheet, ignoring the displayed sheet name of “Source sheet 1”.

I’ve renamed the original sheet with an “_old” suffix and created a new sheet with the same layout but with a the original sheet name.

Excel is smart enough to see this and gives you a choice of sheets – original one with the “_old” suffix or the new sheet with the original “Source sheet 1” name. Saved from certain disaster I bow down again before the genius of Microsoft. Or should I?

A modest man with much to be modest about

We have now reached the inspiration for the title of this blog, and alas and alack, there is no false modesty here. What do I mean by this? I simply mean that as you can see from the above, Microsoft made linked workbooks, at least from the source sheet perspective, goof proof. Believe me, I tried very, very, very hard to break Excel in the course of building the samples for this blog post and I just could not make it happen.

And yet I managed to make a mess of linked workbooks even after being warned that just such an error could occur. How did I manage to do this? Read on so my humiliation can be complete.

Screwing up, step by step

1 – Prerequisites

Environment

I have thus far shown you a very simple set of workbooks and sheets.

Imagine a very complicated set of linked workbooks, with the source being the user set of workbooks that define base budget information and the target being a series of linked worksheets that send data to Essbase. When I say complex, I mean there might be 20 to 70 (yes, you read that correctly) sheets in the source workbook and at least that many in the target. Within each sheet is a link range (think of it as a mapping of source layout to target layout) and then there is another range of linked (this time only within the sheet) formula cells that are selected and sent to Essbase. And before you start tsk-tsking, remember that this is not my process – I am just a caretaker and yes I hate it.

A birth and a death

The business created a new entity. At the same time an old one was closed. This is important.

2 – Getting the request

On the source side, Mabel (hello, Mabel, and yes we are almost to the “I warned you, but you didn’t listen” bit) reused and renamed an existing source sheet for the new entity. This makes sense from her perspective as the formatting, formulas, etc. were already defined. She simply put in a new description, new base numbers, and the existing source worksheet did its magic. Thanks very much Microsoft.

Over on the target side, because there were so many sheets, because I do not really know the entity structure, because this is a manual process, and mainly because I am a dope, I did not follow Mabel’s lead. This was A Bad Thing as we shall see.

3 – Adding the sheet

As the layout of all of the sheets are the same I copied an existing target send sheet – note that this was not the dead entity target sheet but another one. Yes, that was another Bad Idea.

Being the super ultra-clever chap that I am (ahem), I knew that I would have to change the link formulas in the new target sheet and duly did so.

But I also did not delete the old send sheet for the closed entity. This was not a super ultra-clever act and is really the key to the error.

4 – Doubling the data

So what happened? As I showed above, Excel keeps an internal name for all of the source worksheets. A simple sheet rename doesn’t break the links. This is good, right? Right? Wrong.

Not deleting the dead sheet meant that Excel now linked twice to the renamed source sheet, once for the old dead entity and again for the new entity. As there was supposed to be no new data in the dead entity (remember the Cameron is a dope factor) I then managed to double the data in Essbase. Oops.

And this is, btw, precisely what Mabel warned me of.

Ameliorating the problem in the future

Putting aside the issue of me as a user of anything but the most basic of Excel workbooks, how can something like this be avoided?

Excel is not the answer

Excel has auditing features, but it does not (at least as far as I can see) have a simple way to show what the linked worksheets are except by examining each sheet (not terribly useful when there are 70 sheets in play) or by writing a custom VBA macro to try to report the information. Both of these are difficult-ish and time consuming. There has to be a better way.

Power Utility Pak v7

And there is in the form of an Excel utility called Power Utility Pak (PUP). I have to thank Dave Farnsworth (who has “swallowed the anchor” but continues to teach me) yet again for telling me about this product years ago. If only I had it installed on my client machine…

The famous John Walkenbach wrote this utility and it is, in a word, awesome.

Without going into all of the many features of this tool, and there are many, I will focus on the bit that gives you an audit report of linked worksheets.

The Workbook Link Finder is exactly what I should have used.

PUP even gives you a way to display the links:

Given a target workbook with a copied target sheet (just what I did in real life), the following report comes out of PUP:

Looking at the above report, I can easily see that Target sheet and Target sheet (2) both link to the same source sheet. In my example, this is precisely what I did not want to happen, but did.

Check it out for free

J-Walk has a free trial download available here. I encourage you to try it out. I am going to buy another copy and install it (if I am allowed) on my client laptop.

What have we learnt?

I think this can be summarized into a few key points:

Cameron the Essbase hacker is not Cameron the Excel hacker.
Yet Another Manual Process (YAMP) equals Yet Another Chance For Failure which begets Yet Another Stupid Excel Trick
Tools like PuP can make life a little less painful
Linked workbooks are evil

Okay, the last bit is just my opinion, but my goodness a simple Excel error led to a big Essbase data problem.

Be seeing you.

You’d think they’d know better, wouldn’t you?

Over the next three weeks I will be participating in two webinars for ODTUG. Whether this strikes joy, terror, or simply gives you a bad case of avoir le cafard I cannot say.

I will share with you that actually writing the content for these sessions has been a bit of an exercise in pain but I am now all done and if I do say so myself, what I have is not half bad.

All kidding aside, I am very excited about this opportunity to share some of what I know and the subjects are two that are near and dear to my Oracle EPM heart.

The Most Awesome Planning Calculation Manager Hack the World Has Ever Seen

The most awesome? World-beating? That's mighty big talk for a webinar. Is this presentation hyperbole or fact? It's fact--cold, hard, “Just the facts, ma'am,” fact. This webinar deserves those adjectives because it will demonstrate the incredibly clever Planning Calculation Manager hack that Christian Musci and his team invented. It answers the problem the common Planning Focused Aggregation technique could never resolve--Focused Aggregations based on Planning form row and column selections. Did I mention it is a hack? And that it is awesome because it solves a problem that had no solution? And that the hackiness and the awesomeness combine and become a must-see webinar? Maybe I just did.

Join me as I explain the genesis of my understanding of the Focused Aggregation technique, a step-by-step illustration of how it works, its superiority to generic aggregations, the seemingly insolvable problem of Planning not passing row and column selections to Calculation Manager, and finally the beyond-awesome hack that solves this problem. It is an awesome hack; I may have mentioned that previously. Maybe.

With this last bit of the Focused Aggregation puzzle, applications that simply couldn't work, or could only work in a crippled manner through administrative aggregations that robbed Planning of its real-time nature, now quite simply do work. Yes, it's that good and it's easy. All will be revealed in a step-by-step process that will allow you to make your "too big" Planning application "just right.”

Yes, I have covered this subject in this blog and (somewhat) at conferences but this is the first time where I present on the theory behind the technique, show comparisons in performance against other code techniques, and then show off a totally awesome hack that makes it all worthwhile all at once. This is a deep, deep, deep dive into the technique. If you are not using this technique, or not using the latest version of it, you really should block off the hour to hear about how this works.

The when and the where

25 February 2014, 2 to 3 pm, Eastern

Getting (even more) Serious about Data Quality and Governance

A few things I must mention right off:

This is a webinar panel, not a traditional presentation like the above one on Focused Aggregations in Calculation Manager.
I am but 1/3 of the team. Ron Moore and Joe Caserta are the heavy hitters. Think of me as the comic relief.

Nothing is more important than data quality. But if the steps to insure high data quality aren’t fast and easy people won’t do them – or at least they won’t do enough of them. It was always a difficult job and it consumes a lot of resources even with traditional data sources such as ERP that are relatively well behaved. Now analytics is spreading to more users and to data that’s far less well behaved. What should we be doing and how can we make it as fast and easy as possible?

In this webinar we will put those questions to our panelists and we will invite your opinions and questions. Some of our topics will include:

Is data quality really a problem? Where and how much?
Who has responsibility for data quality?
What techniques can we apply at the data source?
What techniques can we apply within Essbase and Planning?
Can we adopt some “simple stupid rules” for DQ?
What is the role of documentation?
What documentation is effective and worth the effort?

The when and the where

11 March 2014, 3 to 4 pm, Eastern.

That’s all for now

The topics are interesting, the content is good, and as always ODTUG provides all of this to you for free, nothing, zero, zilch, etc. How can’t you win?

Join us, won’t you?