SAFMQ 0.8.5 Released today

Hot off the press, SAFMQ 0.8.5, with updates for MacOS X, FreeBSD 11, Fedora Core 28, and test compilation for 64 bit Windows.  The 64 bit installer for windows is on it's way...

Check out the latest files: https://sourceforge.net/projects/safmq/files/SAFMQ/SAFMQ%200.8.5/

Grab documentation on the project site:  http://safmq.sourceforge.net

Store and Forward Message Queue

I'm looking back through my blog posts and it seems I've never posted about my long lived open source project SAFMQ.

Just recently I've had some time to get back to it... As I'm looking at the traffic at Source Forge, and I'm a little downtrodden.  Back in the day (prior to 2014) there was such a thing as FreshMeat which became Freecode and died.  Back then you could advertise your open source project, and you'd get a bunch of traffic sent your way.  I'm not really sure why Freecode/FreshMeat went away, but the official reason was "The Freecode site has been moved to a static state effective June 18, 2014 due to low traffic levels and so that folks will focus on more useful endeavors than site upkeep."  I'm not sure what that really meant, because there were nearly 40 packages announced that day...

Anyway, I'm back at, and some of the things I've been looking into are making it Mac OS compatible, improving some of the not-before/not-after X.509 cert management, and gathering performance tests on a bunch of platforms for comparisons.  More to come

NPM and External Dependencies for PowerBI Custom Visuals

The custom slicer I built for Power BI has a few advanced UI features, clickable DIVs, a slider control, text input with validation.  It may have been possible to develop all of those interactions myself, but why intent the wheel.  jQuery and jQuery UI really add all of the features and simplify a lot of the coding.

Here's a step-by-step process that I used to get jQuery and jQuery-UI into my visual.

1. Start with Use Developer tools to create custom visuals.
2. Then add jQuery and the strong types to be TypeScript friendly:
1. npm install jquery
2. npm install "@types/jquery"
3. Add jQuery UI (this was the hard part, see below).

I'm new to Node.js projects (do I capitalize Node.js, yes?), and I'm still getting familiar with the overall structure.  The thing I'm getting off the bat is that the node_modules folder is special.  For one, it should be 100% managed by npm.  That means that stuff placed there shouldn't be committed to your code repository (I'm not there 100%, but I'm getting close). 

Full disclosure, I started by putting some non-TypeScript code there that I had written (CSS, custom JavaScript, etc.).  But it's not an npm package, so it shouldn't be there. I moved it to my src folder. Next, I tried and tried to use an npm package for jQuery UI.  I tried the jquery-ui package, but it has to be built.  Then I got some new info and tried the jquery-ui-dist package.  But, I'm getting an error when compiling.  Near the very last line of jquery-ui.css there's a rule:

filter: Alpha(Opacity=.3);

But, Node.js doesn't understand that rule and throws the error "error  LESS  style/visual.less : (1307,23) Could not parse alpha".  I did some searches and the only way to fix it is to modify the code.  I don't want to modify the code in  node_modules.

My fix was to download the vanilla version of jQuery UI.  Then cherry pick only the assets I need to be successful.  I grabbed jquery-ui.js, jquery-ui.min.js, and the minimal items from the css folder in the distribution.  For the timebeing they are in a jquery-ui folder in node_modules.  I know I just got done writing that this was bad, so my next step will be to move them to the src folder.  Optimally, it would be great if I could just use the jquery-ui-dist package, but it seems that I can't, yet...

Removing Document Edit Protections from Word DOCX files

Ever wanted to edit a Word document that had editing disabled?  Typically you need to have a password to be able to edit the file, but if you really want to get rid of it, there's a pretty straight forward process.

1. Unzip the DOCX file (it's just a ZIP file after all) into a directory some where.
   • This produces the following:
   • [Content_Types].xml
   • _rels (sub directory)
   • docProps (sub directory)
   • word (sub directory)
2. Using your favorite text editor (I used VIM), edit the file word/settings.xml
3. Remove the XML tag:  <w:documentProtection/>
4. Zip up the three (3) directories and the [Content_Types].xml file and make sure the extension on the zip file is DOCX.

You're ready to go.  You've now removed the protections on a Word Document.  Be careful with your new found powers, young padawan.

Power BI Custom Visual Design Considerations

Back at the Power BI custom slicer... We ran into an intermittent bug that would happen when you first open a report and when you switch between tabs.  The root cause comes from a feature where the first or last item is automatically chosen, something handy when you want to see the latest data based on a range of dates.

Some background on Power BI Custom visuals, they are TypeScript (JavaScript) based, and are defined by the interface IVisual.  As of v1.11.0, the IVisual interface has three methods:

• update(options: VisualUpdateOptions): void
• destroy?(): void
• eumerateObjectInstances(options: EnumerateVisualObjectInstancesOptions): VisualObjectInstanceEnumeration

There's also a constructor that receives a VisualConstructorOptions object, and a bunch of methods that aren't documented, yet.  Turns out that update() is called immediately after the constructor, and after every call to ISelectionManager.applySelectionFilter() (APF here on, which changes the filters applied to any connected visuals).  When APF is called and the selection is the same as it was before, there's a good chance that it won't trigger an update, which is nice because people like to double click stuff.

Here the slider sent an on change when the selected value was updated programmatically.

The kicker is when your custom visual changes the selection and calls APF during a call to update().  Mine did, on every call, and the only thing keeping it from an infinite loop of updates was the under-the-covers check to see if the selection was the same.  But what if there was a delay in the selection updates, and it caused a cycle between an old selection and the one that was just forced?  Infinite loop...

I tried all kinds of things.  First, I thought that there was a direct path between APF and update(), so I put some Boolean locks around the code calling APF to avoid stack overflows and stop the loop.  But, even though it looks like it on the stack inspector, the selection changed events are detected in some sort of async loop instead inline with stack execution.  Then I tried to make the code as clean as possible, but that didn't help.  The piece of information I was missing?  The very first call to update() after the constructor restored the previous selection filter w/o having to call APF.  Support on the Power BI board, said to never call APF from update(), because it can cause an infinite loop, and testing showed that the previous selection was always pre-choosen when the report loads (as long as you store the selection under the property "general" when it changes, more on that later).  So don't call APF from update(). 

But I really wanted to pick the first or last item by default.  The change was to be sure to never call APF from update() more than once.  I was already detecting the first call to update() after the constructor, then it was just a matter of:

• If it isn't the first call to update() don't call APF.
• If it's the first call to update() and only if the current selection isn't the one you want call APF.

In this version, onChange() isn't called during a call to update()

With this change to the logic, APF could only be called w/o user intervention once after the report loads, and so no loop.

Coding PowerBI Custom Visuals

In my last post I discussed a need for a new slicer visual for Power BI.  There's only one slicer visual built into Power BI.  It's pretty good in most cases.  The Power BI Slicer visual provides the following selection modes:

• List - for all types
• Dropdown List - for all types
• After, Before, Between, Relative - for dates
• Between, Less than or equal, Greater than or equal - for numeric values

Pretty powerful set of choices.  Plus Power BI provides special handling for the slicer visual.  When you cut and paste a slicer from one page to another page, Power BI offers to automatically synchronize choices.  And if you didn't go that route, the View tab in the Ribbon has a checkbox to show the "Sync slicers" pane.  Selecting the Slicer visual in the report area will update the Sync Slicers pane, and show you all of the other pages where a Slicer is using the same field to filter content on the page.  Overall a valuable feature for reports with multiple pages of related information.

But... What if you have a need to filter a report by a single discrete value out of a large set of values?  The List/Dropdown List isn't a very good option, and if it's dates, the range selection can be clumsy...

So with that said, I jumped right into building my own slicer.  Requirements:

• Work with large set of discrete values
• Provide forward and backward paging
• Provide a thumb to drag
• Provide a text box for direct entry

Also, it should have some validation on text entry and paging to keep it in bounds.  It would also be nice to remember the last selection, or default to the first value or last value when reloading the report.

This was the easy part.  Next up, I'll write up the steps I took to put the feature together.

Power BI Custom Visuals

The last few weeks I've journeyed into the strange realm of PowerBI Custom Visuals.  My goal was to create a new slicer that picks a single value from large set of values.  We have a need to look at daily reported values.  The Range picker won't work well because it's difficult to pick single day.  The drop down works well when there's only a handful of items.  But, what if you're working with a timeline, especially when there will be hundreds or thousands of choices.

In, comes the custom visual.   I blended a jQuery UI Slider with a text input box, and two spans styled as buttons.  The user can use the increment and decrement buttons to move to the next item, click the bar or drag the thumb to zoom to a location, or enter the exact value they are looking for.  There's some validation built in.  For instance, manually entered values are checked against the list of possibles.  The user can't make the control increment or decrement pas the end of the sets...

Next post, getting into the dirty work of implementing the visual.  The whole thing is a TypeScript project following the Node.js patterns.  NPM works great to add some of the common features, but the uncommon ones?  I may have not followed the specs perfectly when including custom JavaScript (jQuery UI + code to implement the combined UI elements), but it's working at the moment.  I'll be digging into how exactly to bring these custom features into a project correctly, as it appears that "non project" code should be include through NPM...