Tuesday 31 October 2017

Using R Sample (Base64Image) Plugin in Oracle DVD

This is in continuation of the earlier blog on Oracle Data Visualization with Advanced Analytics, let me go a step further and try to use a R Code in Oracle DVD.

To begin with Oracle DVD is built with R and Java, and for the Analytics options like Clusters, Outliers etc we can find the corresponding codes in 'Oracle Data Visualization Desktop\OracleBI1\bifoundation\advanced_analytics\script_repository' folder.


For this demonstration, I am going to use Base64Image plugin that displays a base64 encoded image directly in Oracle DV.Image is assembled from chunks of encoded strings. It requires three inputs: image id, image parts sequence and image parts.

This plugin is available at Oracle Analytics Library along with lot of other plugins, may be we can try to explore them one by one later.


Once we have downloaded the plugin, we go to console and upload the custom plugin.


Once the plugin is successfully uploaded we can view it in the list of visualizations.


For the purpose of this demonstration, let us use Sample Order Lines data. Once we add the Base64ImgViz Plugin, we see it is asking for Image ID, Image Part ID and Image Part.


Now before we proceed any further we need add the custom R script in Script Repository. And for that I am going to use the xml below :

<script>
       <scriptname>obiee.RImageEncSplit</scriptname>  <version>12.2.1.0.0</version>
       <inputs>
              <column>
                     <name>Prod</name>
                     <nillable>NO</nillable>
                     <required>YES</required>
              </column>     
              <column>
                     <name>Sales</name>
                     <nillable>NO</nillable>
                     <required>YES</required>
              </column>
              <column>
                     <name>Profit</name>
                     <nillable>NO</nillable>
                     <required>YES</required>
              </column>
              <column>
                     <name>Q_Ordered</name>
                     <nillable>NO</nillable>
                     <required>YES</required>
              </column>
              <column>
                     <name>Shipping_Cost</name>
                     <nillable>NO</nillable>
                     <required>YES</required>
              </column>
       </inputs>
       <outputs>     
              <column>
                     <name>img_id</name>
                     <datatype>varchar(10)</datatype>
                     <aggr_rule>none</aggr_rule>
              </column>
              <column>
                     <name>img_part_id</name>
                     <datatype>varchar(10)</datatype>
                     <aggr_rule>none</aggr_rule>
              </column>            
        <column>
            <name>img_part</name>
            <datatype>varchar(2000)</datatype>
            <aggr_rule>none</aggr_rule>
        </column>
        <column>
            <name>Metric</name>
            <datatype>double</datatype>
            <aggr_rule>sum</aggr_rule>
        </column>
       </outputs>
       <options>
         <option>
              <name>dummmy</name>
              <value>100</value>
              </option>     
         </options>
       <scriptcontent>
  <![CDATA[
function(dat,dummy) {
 library(base64enc)
 #################### HeatMap Generation ######################
 for (i in 2:ncol(dat))
 dat[,c(i)] <- as.numeric(dat[,c(i)])
 hm_dat <- as.data.frame(aggregate(dat[,c(2:4)], 
            by=list(Product_Sub_Category=dat[,c(1)]), FUN=sum))
 df <- as.matrix(as.data.frame(lapply(hm_dat[,c(2:4)], as.numeric)))
 metrics_matrix <- as.matrix(df)
 row.names(metrics_matrix) <- hm_dat[,c(1)]
 fpath = tempfile()       
 # setup tempfile to capture the heatmap image
 png( fpath, width = 600, height = 600 )
 hv <- heatmap(metrics_matrix, col = cm.colors(512), scale="column",                        
          Rowv=NA, Colv=NA, margin=c(5,10),xlab = "",
          ylab= "Product Sub Categories",main = "Metrics heatmap")
 dev.off()
 # Convert the image into a base64 encoded string
 p <- base64encode(fpath)                                                               
 # Break the image into substrings and number them
 s <- substring(p, seq(1, nchar(p)-1, 2000), seq(2000, nchar(p)+2000, 2000))            
 # Capture all the fragments into a data frame
 o <- data.frame(img_id=1,img_part_id = substring(1000+1:length(s),2,10), img_part = s) 
 o$Metric <- 1
 return(o);                                                                              
 }
]]>
       </scriptcontent>
</script>

This script is divided into :
  1. Input Columns
  2. Output Columns
  3. R Script
    • Taking the data
    • Set up the temporary file to capture the image
    • Generate the image for heatmap
    • Convert the image into base64 encoded string  

In the script we are using library(base64enc), and to use that we need to install base64enc package.


Alternatively we can directly download the package from Cran Site, and unzip it and place it in the R Library.


And we can test it also.

We can ignore the warning message, it is appearing because my R is 3.1.1 and it will not cause any problem.

Now let us go back to DVD and build 3 calculated columns to be used in Base64ImgViz Plugin

  1. Img_Id : EVALUATE_SCRIPT('filerepo://obiee.RImageEncSplit.xml','img_id', 'Prod=%1;Sales=%2;Profit=%3;Q_Ordered=%4;shipping_Cost=%5', XSA('weblogic'.'Sample Order Lines')."Columns"."Product Sub Category",XSA('weblogic'.'Sample Order Lines')."Columns"."Sales",XSA('weblogic'.'Sample Order Lines')."Columns"."Profit",XSA('weblogic'.'Sample Order Lines')."Columns"."Quantity Ordered",XSA('weblogic'.'Sample Order Lines')."Columns"."Shipping Cost")
  2. Img_Part_Id : EVALUATE_SCRIPT('filerepo://obiee.RImageEncSplit.xml','img_part_id', 'Prod=%1;Sales=%2;Profit=%3;Q_Ordered=%4;shipping_Cost=%5', XSA('weblogic'.'Sample Order Lines')."Columns"."Product Sub Category",XSA('weblogic'.'Sample Order Lines')."Columns"."Sales",XSA('weblogic'.'Sample Order Lines')."Columns"."Profit",XSA('weblogic'.'Sample Order Lines')."Columns"."Quantity Ordered",XSA('weblogic'.'Sample Order Lines')."Columns"."Shipping Cost")
  3. Img_Part : EVALUATE_SCRIPT('filerepo://obiee.RImageEncSplit.xml','img_part', 'Prod=%1;Sales=%2;Profit=%3;Q_Ordered=%4;shipping_Cost=%5', XSA('weblogic'.'Sample Order Lines')."Columns"."Product Sub Category",XSA('weblogic'.'Sample Order Lines')."Columns"."Sales",XSA('weblogic'.'Sample Order Lines')."Columns"."Profit",XSA('weblogic'.'Sample Order Lines')."Columns"."Quantity Ordered",XSA('weblogic'.'Sample Order Lines')."Columns"."Shipping Cost")
What we are doing is passing the input columns and getting the output columns as stated in the xml.

Once the calculated columns are done we can add them in the Base64ImgViz Plugin visualization and the result is like,


We can add filters also to further slice / dice the data.


And that's it. May be next we can try to explore some more R scripts or DVD plugins.

Saturday 28 October 2017

Oracle Synopsis Lens

Few days back I came across this feature called 'Lens' in Oracle Synopsis. What it does is really cool, it can read any data and can create BI Analysis / Charts on that automatically. Let me try to demonstrate one of such cases.

I have created a sample data out of my grocery list.


Probably next time I will try to use the grocery list itself ... 😏.

I open the Oracle Synopsis in my mobile (in case you want information on how to download or use Oracle Synopsis, please refer to my earlier blog on Oracle Synopsis). On the home screen, there is the + sign and once you click that, you get the 4 options for creating a new analysis.


In this case I will go with the Lens.

I click on the Lens and point my camera towards my laptop screen showing the excel data.


And I also 'Enable chart'. Few minutes of holding the mobile like that and I get a complete analysis, just like that.


Few things that I have noticed and would like bring up :

  • Performance wise it takes a bit of time to come up with the analysis. 
  • Actually it tries to put the data into different form of visualizations while preparing, but most of those visualizations are not used.
  • Some amount of manual editing is required once the analysis it done, to make it ready for sharing.
I have tried to capture it using a small demo video also.


Hope it will be useful, I will try to explore and post new features /tools.

Implementing & Testing Row Level Security in Power BI

I have suffered a great deal of pain while implementing and more so while validating Row Level Security in Power BI. Let me try to capture a...