When the tagline becomes the brand

While strolling through the supermarket, I was surprised by Kellogg’s new cereal packaging.  You can see the new Rice Krispies package on the left next to the old one on the right.  In the new packaging the brand tagline is so much more prevalent than the product name.  Perhaps this is how kids are asking their parents for sugary breakfast cereals…Mom, I want some Follow My Nose..

I’m interested in seeing how this works out for them, but right now I have more pressing matters.  My check engine light just came and I have to get to the three best friends my car ever had.



Should businesses organize marketing teams by channel?

Many large organizations break their marketing teams up by channel.  There’s the email marketing team, the social media team, the web team, the traditional media team etc.  Most often, this structure grew organically based on skill sets; one person understood email, another SEO/SEM, and eventually they found someone who understood social.  As the business grew, these individuals enlisted support and grew into teams.  To a certain extent this alignment by area of expertise makes sense, but it also causes as many problems as it solves.

This division of labor within marketing no longer reflects the customer experience.  It has been a looooooong time since customers primarily interacted with a brand through a single channel.  Customers are product and brand centric to a much greater degree than they are channel centric.  9 times out of 10, structuring the team by marketing channel leads to incoherent marketing reporting, and poor marketing attribution.  Each team will measure their marketing success in the language of the channel, and each team will take sole credit for the conversion.  Senior management won’t be able to compare marketing efforts because they are using different metrics, and won’t believe the results anyway, because the conversion metrics are twice the actual sales figures.

To compound the problem, dividing the marketing team by channel causes the higher levels of the organization to think of marketing channel as the primary division for marketing spend.  Instead of thinking “we need to optimize the conversion funnel for new customer acquisition”, they think, “I need to cut total marketing budget by 10%, and so the social media team needs to cut one headcount.”  This further disincentives the channel teams from seeking the most accurate campaign reporting.

So what structure would work better?  Depending on the size of the organization and division of products, it is usually better to organize the marketing team around either product line, or the customer lifecycle stage.  These divisions are a much more accurate reflection of the customer experience.  For large multi-brand companies where there is not a lot of cross-over between brands, a brand centric organization may work best.  For organizations with a more homogeneous product line, dividing marketing into Acquisition, Development, and Retention teams may be a better strategy.  Separating the reporting and analytics group from the execution teams provides can further promote a more balanced view of marketing performance.

How is your marketing team organized?  Do you think it works?  Am I full of it?  Comments are always appreciated.


Philly Music Hackday: Are we getting happier?

I thought I’d use the Philly Music Hackday as a chance to dive into the Echonest API and get my hands wet with D3.js.  The charts below look at the valence (roughly = mood) and energy of the number 1 charting singles on May 20th from 1941 to 2013.


The “happiest” songs are those with high energy and high valence.  The valence and energy numbers are added for the first chart.  This assumes equal weighting between valence and energy, and I’m not sure that’s the case, but it is a start.  The dip in the middle of the chart is Roberta Flack’s 1972 hit, “The First Time I Saw You Face“.



The second chart breaks down results by decade.  Most decades have at least one outlier, but there is definitely a trend up and towards the right since 1990.



I had a great time at the hackathon and am looking forward to doing more with both the D3 and Echonest libraries.



My 2013 Marketing Analytics Prediction

Over the past decade, the focus of data analysis in top tier marketing organizations has developed from reporting of past campaign performance to predictive analysis of future campaigns. However, limitations to the speed of analysis and a high degree of required technical expertise have caused the focus of marketing analysis to remain narrow, directed primarily towards specific marketing activity outcomes. Innovations in the ease of data integration, quality of visualization, and automation of low-level analysis will cause predictive campaign analytics to no longer be a key differentiator between marketing organizations. Instead, those organizations with the best analytic skills will shift the level of analysis higher and further into the future to opportunities with broader implications to the overall company strategy such as product and market development, similar to the types of analytic activities currently performed at the best hedge funds. The ability to rapidly synthesize the increasingly varied sets of data will become the new standard of marketing analytics excellence.

Knoetry presenting the Next Wave in Marketing Analytics

I will be giving a presentation on the next wave in marketing analytics at The Macrotheme Business and Economics Conference in Paris on October 16th. It is turning out to be a great conference this year.

Talend Connect live from Boston

It’s a full house here at the Millenium hotel in Boston. So far, Bertrand has done a great job layin out Talend’s position and vision for the next 5 years.

Yves led us through a humerous and rapid fire presentation on the evolution of open source software. 117 slides in 18 minutes. Wow!

Obuli Navakoti from Bank of America has just laid it down with how a simple enterprise class Talend installation can make big improvements in processing speed and cost savings.

Break time. More to come.

Back from break.

Mark Jeffries gave an entertaining talk, sans PowerPoint, on business speaking. Boy, that man is fond of triumvirates. There were many, but the main one was that 1) We are all in sales 2) Everything you do and say in a conversation adjusts the balance of favor, and 3) oops I already forgot. LWAR?

Back from lunch.

Joe Caserta, who wrote te book on ETL, gave a great talk on the growing use of Big DATA.

Ed Ost from Talend and Ronak from BonitaSoft covering BPM and ESB integration with Talend.

Tony from NEO Ogilvy. Acquisition Hygiene Usage. Hygiene is needed, so if you spend all your time acquiring, you jave less time for usage.

Jeff Kelly from Wikibon covered the main big data players and the challenges in adoption. Mainly a lack of data scientists with big data skills. “Big data is about answering the questions you didn’t know you had.”

Fabrice from Talend shared the roadmap for Talend through 2013. Even faster ETL performance. W00t!

Last session. Mark Jeffries hosted Bertrand, Jeff, Tony, and Miguel from BonitaSoft in a discussion focusing on big data. The big takeaway is that there is still work to be done in making Big Data tools ready for less sophisticated users (non-programmers), but that work is being done with Talends help, and it is not just hype.

Time for Talend-tinis!

Talend Tips for Database Developers, Part 2 – Data Type Conversion Errors

Part 1 of this series covered ways to avoid the dreaded NPE.  This installment covers a few mitigation strategies for data type conversion errors –  namely, using Talend’s tConvertType component, creating data type conversion functions, avoiding PITA data types like BigDecimal when appropriate by modifying Talend’s default Java type conversions.

Note: This entire series assumes you are using Talend to generate Java jobs, not PERL jobs.



What are data type conversion errors, and why do they occur?

Because most Talend ETL activities are actually taking place in memory and processed via Java, rather than in the native database, Talend converts database data types to Java datatypes for processing. Java is a strongly typed language.  That means it does not do implicit data type conversions and in general, it is much pickier about data types than a typical database.


Data Type Conversion errors occur when there is a difference in data types between an input and output row, or between the expected inputs and outputs of a function.  They are most likely to occur in the tFileInput<xx>  or tMap components.  If the error occurs during file load, check for descrepencies between your file schema and the actual data in the file.  If the error occurs in the tMap, try one of the methods below.


Using tConvertType

Talend supplies the tConvertType component to assist users in performing data type conversions without resorting to code.  I usually use this component directly after the file or db input, but it could go anywhere in the job.

The input and output schemas for the tConvertType are the same, with the exception of the data types.  When editing the schema, the input data type should match the data types of the component preceding tConvertType.  Change the output data type to the type required in the next step of processing.  The component is useful for converting between numeric data types (long to int, BigDecimal to Double), and between numeric and string types.  However, it is much more limited for Dates.  I usually use one of Talends built-in date functions for these types of conversions.


Who’s afraid of the Big (Bad) Decimal? – I am.

In Java, there is a BigDecimal class which operates as a pseudo data type.  It is not a primitive data type like a float or a double.  BigDecimal does have some advantages, such as not losing precision or auto rounding, but they come at a rather large cost for most developers.  The biggest issue is that a BigDecimal is immutable.  That means normal arithmetic functions like +,-, x, / cannot be used.    Instead, class methods of .add(), .substract(), .multiply(), and .divide() are used.  But wait, it gets worse.

Lets say you have the BigDecimal, monthyFees =123.456789, which represents monthly fees, and you want to multiply by 12 to get your annual fees.  We already know that monthlyFees x 12 won’t work.

How about monthlyFees.multiply?

No.  That won’t work either.  Why?  Because 12 is an Integer, not a BigDecimal.  To make this work, 12 also needs to be converted, either to a BigDecimal or a Double, resulting in :

<code>monthlyFees.multiply(Double.valueOf(12.0))  </code>

To me, that seems like a lot of hoops to jump through to convert monthly fees to an annualized total.  If you have a lot of decimal places, very large numbers, and precision matters, then use BigDecimal.  Otherwise, don’t.


Modifying the DBType to TalendType mappings

If the data model you are connecting to frequently uses the Numeric data type, you will soon suffer from Death by Thousand Data Type Conversions.  Fortunately, there is a way to change the default data type conversions in Talend so that you can reduce the number of explicit conversions programmed.   Talend stores mappings of database data types to talend data types in XML files.  You can access and edit these files by selecting Window > Preferences > Talend > Specific Settings > Metadata of TalendTypes



Select the mapping appropriate to your database and then click Edit.  Scroll down to the dbToTalendTypes section and search for <dbType type=”NUMERIC”> – highlighted in the image below. Change the line beneath from talendType type="id_BigDecimal" default="true" to talendType type="id_Double" default="true" .

Click OK to save the changes.  Now Talend will change Numeric db types to Doubles by default and BigDecimal can be used for fields only when necessary.


Creating custom functions to convert data types

One other challenge for database developers is that the syntax used to convert a data type in Java is dependant on the source data type in addition to the target data type.  Although most of the time something.toString() will convert a value to a String and DataType.valueOf(something) will convert a value to a specific numeric data type, the exceptions will soon rear their ugly heads.  If your development team has strong experience in a specific database platform, the learning curve for data type conversions can be shortened by creating a custom set of functions that mimics database syntax.

The example below uses Oracle style calls to convert data types to Long.  Notice that there are several version of the TO_LONG function, each accepting a different input data type.  This is called function overloading and allows the Talend user to call the function using identical syntax for any of the specified input data types.  This is extremely useful for data type conversion functions, but can be used for any function.
* TO_LONG: converts a string, double, or integer to a Long data type
* {talendTypes} Number
* {Category} Data Type Conversion
* {param} Long(inValue) input: the number to be converted to a double
* {example} TO_LONG("123456789101112") returns: 123456789101112
public static Long TO_LONG(Double inValue) {
if (null==inValue) {
return null;
else {
String inString = inValue.toString();
return Long.valueOf(inString);

public static Long TO_LONG(Integer inValue) {
if (null==inValue) {
return null;
else {
String inString = inValue.toString();
return Long.valueOf(inString);

public static Long TO_LONG(String inString) {
if (null==inString) {
return null;
else {
if (StringHandling.IS_NUMERIC(inString)==true) {
return Long.valueOf(inString);
else return null;

Once the set of functions has been created as a class in Talend, it can be distributed and used in all projects and jobs by Talends Export and Import functionality.  Creating the class is more work up front, but if you have a large group of developers with little Java experience, it can be a great time saver.

That’s about it for this long winded post.  The next part in the series will focus on recreating SQL database functions in Talend.


Talend Tips for Database Developers, Part 1 – Null Pointer Errors

Talend Open Studio is a great tool.  It has made my job easier and less error prone, and it’s free! What more could you want? That said, there are some sticking points for new users, especially those that come from the database development side, as opposed to Java developers. Most of the difficulty I see database developers having with TOS stems from Java, and in particular – Null Pointer Errors (or NPEs) and datatype conversion errors. The good news is that these errors can be drastically minimized with just a few tips.

Stored Procedures    Lead us not into temptation. Just tell us where it is; we’ll find it. – Sam Levenson

First, I want to put out a word of caution about what I think is the “wrong” way to solve for these problems. When you combine a pile of existing SQL code or stored procedures, and frequent Java NPE and data type conversion errors, it is easy to see why an organization would be tempted to use Talend strictly as an ETL flow diagramming tool. What I mean by this is that the developer will use Talend to call a stored procedure in the database, and then on completion of that component, kick off the next stored procedure.

Talend Stored Procedure Flow


Hey this is great, now we have a pictorial view of our SP based process. Isn’t life wonderful? Although having a a visible flow is beneficial, this approach does not take advantage of some of the most significant benefits of switching to an ETL tool. This approach loses the layer of abstraction between the code and the data model. If a new field is added, the code has to be changed by hand. It also diminishes the ability to understand the relationship between the business rules and the data transformations. If something goes wrong, you are back to digging through the code to understand the error, although at least you may have a better idea of where to start. Lastly, development speed is not improved much by this method because so much code is still being written by hand. There are a few instances where I recommend keeping the stored procedures (such as when significant user defined functions are part of the operation that would be prohibitive to recode, and/or the data is not migrating from one server to another), but these should be the exception, not the rule.


Null Pointer Errors

So now that we’re all on the same page about overusing stored procedures calls from Talend, what can be done to make the actual work of transforming data in Talend easier, or at least more intuitive to a database developer. One of the big differences between databases and Java based programs is the way nulls are handled. If a null is run through most database functions, a null is returned. Try this with standard Java methods, or even the built-in Talend functions, and you’re likely to get a Null Pointer Error. Oooh, nasty. There are a few ways to avoid NPEs: convert nulls to a default value on the input, switch the order of constants and variables in your function calls, and either modify Talend’s existing functions or create a new class that handles nulls.

Taking a look at the first approach, you can define default values to replace nulls either in your source system, or using the “Default” field in your input component’s metadata.

Talend Edit Schema Set Defaults

I prefer to use this approach as my starting point whenever possible. It is the most reliable way of reducing Null Pointer Errors. If your organization does not already have standards in place for setting default values of different data types, get together the appropriate people and make some. If your default integer is sometimes 0 and other times -1, things are going to get confusing really fast.

The next approach to handling NPE’s, and this is really just a supplemental tip, is to use a constant or context variable on the left side of your string comparison calls.
First, let’s look at the wrong way to use .equals


This code says if myString is ABC, then return DEF, otherwise return myString. The problem with this call is that .equals() is a method of the String data type. If row1.myString is null, then it is not a string. In essence, Java is interpreting the code like this:


Because there is no .equals() method for null, a Java error will return. A better way to make the comparison would be like this:


Now we know that “ABC” will always be a string, so the .equals() method will always be valid for the data type. In many cases, “ABC” represent some value that will be re-used many times in a project. In these instances, use a context variable. Just remember to follow the format of having the fixed value calling the .equals() method.

In earlier editions of Talend, NPEs could also show up in calls to the built-in Talend functions, especially in the StringHandling class, because they usually did not handle nulls. Fortunately, newer versions (version 4.2 forward) of TOS have null handling in the StringHandling and TalendDate classes.

If you are running an older version of Talend, you could use inline if statements to test the values for null before calling the function.

null != row1.myString?StringHandling.subString(row1.myString, 0, 3):null

This works as expected, but can be a pain to manage. Imagine having to use this logic nearly every time you call a function!  You can however.  Or better yet, if you want this type of null handling to be routine, import the new classes to replace the existing System routines.

Hope these tips have been helpful.  Got another tip for reducing NPE’s in Talend? Let’s hear it!   In the next part of this series, we’ll look at  ways to minimize datatype conversion errors.

What would you change if you were measured on engagement instead of ROI?

What would you do differently if your performance was measured by how much your customers benefited from your product or service instead of how much you profited from your customers?

Let’s say you are a radio station with a paid membership program. The program offers outstanding benefits, including discounts at many local stores and marketing partners. As part of the program sales pitch in your membership drive, you explain how the membership could easily pay for itself with just a few purchases. Sounds great. Sounds worth it…sign me up. At this point, the radio station marketers say “Woohoo! We have our $120.” and then forgets about you. They’re on to the ext acquisition cycle. About a month later, I get my membership card and a free gift. About 11 months after that the next membership drive starts the cycle all over again.

Now let’s be clear. The membership card was good for all of the discounts that the radio station promised. Is it their fault that I didn’t use the card? Yes. Or more appropriately – that’s not the point. The next membership drive, how much value do you think I will place on the membership program? How likely am I to renew at the same rate? As an alternative, let’s say that along with the membership card I received an invitation to a members only event – something that got me using the card right away. And on top of that, if I didn’t use the card for a period of time, I received email reminders to get out in the world and take advantage of the discounts. And maybe the emails or POS transactions showed me how much I have saved in the last year by being a member (Barnes & Noble is great at this). These are all things marketers could do to increase my valuation of their program and my likelihood to renew.

Now imagine that I used the program all the time. How many people would I tell about it? How would it change my perception of the value of the program. I could become a real advocate. What if your goal was strictly to create brand advocates? And I don’t mean bribing customers people to refer a friend. Those are not brand advocates – they are low level affiliate marketers. What kind of impact could 1000 brand advocates have on your social media marketing strategy?

Of course you can easily substitute other membership or subscription services – see gyms, magazines, online newspapers etc. But this is also applicable to products. You sell car parts? Get your customers to post photos of how those new rims look on their car. Clothes? What did they pair those new shoes with?

If you want to go from high value customers to brand advocates, you need to push beyond the point of sale to encourage use and engagement.

Your Customers’ New Years Resolutions

Forget about your New Years Resolutions for a minute. What are your customers’s New Years Resolutions. What do they want to accomplish this year, and how can your company be a part of that? The Washington Post lists some perennial favorite resolutions. How can your business help someone get in shape, learn a new skill, eat better, or spend more time with their family (or less time with their extended family – as many feel by the time January rolls around)? Let your customers know how you can help. Let’s face it, we all need help keeping our resolutions on track.