Looking for a Sales and Marketing Expert

Sales and Marketing Expert for Data Analysis Tool (Contract)

Sales and Marketing Expert
Data Analysis Tool

We are developing tools and algorithms to automate complex data analysis, reducing costs and increasing results.

Complex data analysis is a multi-billion dollar business. Major data analysis tool makers alone report revenues totaling over $4 billion per year: SAS Institute ($3.2 Billion), IBM SPSS ($0.3-1.0 Billion), MathWorks ($850 Million), Wolfram Research (at least $40 million), and a number of less well known smaller firms. Medical businesses, financial firms, and science and engineering organizations spend billions of dollars per year on these tools and the salaries of the analysts, scientists, and engineers performing the analyses.

Complex data analysis increasingly determines the approval of new drugs and medical treatments, medical treatment decisions for individual patients, investment decisions for banks, pensions, and individuals, important public policy decisions, and the design and development of products from airplanes and cars to smart watches and children’s toys.

State-of-the-art complex data analysis is labor intensive, time consuming, and error prone — requiring highly skilled analysts, often Ph.D.’s or other highly educated professionals, using tools with large libraries of built-in statistical and data analytical methods and tests: SAS, SPSS, MATLAB, Mathematica, Scientific Python, the R statistical programming language, Excel and similar tools.  Salaries and overhead for these analysts range from $40/hour to $200/hour (using a 25 % overhead rate), sometimes even more.

Total Cost of Analyses ($50/hour)
Analysis Duration (Hours) Total Cost

2 weeks ( 80) $4,000
2 months (320) $16,000
6 months (960) $48,000

Results often take months or even years to produce, are often difficult to reproduce, difficult to present convincingly to non-specialists, difficult to audit for regulatory compliance and investor due diligence, and sometimes simply wrong, especially where the data involves human subjects or human society. Many important problems in business and society remain unsolved despite modern computer-intensive data analysis methods.

A widely cited report from the McKinsey management consulting firm suggests that the United States may face a shortage of 140,000 to 190,000 such human analysts by 2018: http://www.mckinsey.com/business-functions/digital-mckinsey/our-insights/big-data-the-next-frontier-for-innovation

Please see our web site http://www.mathematical-software.com/ for more details.

We are seeking a sales and marketing expert with both technical and sales/marketing experience in data analysis tools such as SAS, SPSS, Matlab, etc. to develop a specific, actionable sales and marketing plan including a list of prospects and/or marketing channels and a projected budget for our product release scheduled for Q2 2018.

Requirements

o At least three years of hands-on experience in data analysis. Graduate research experience leading to a Ph.D. is acceptable. Need not be current experience. We are looking for a sales and marketing expert who started out doing data analysis before transitioning to sales and marketing.

o At least three years of recent paid professional experience in the sales and marketing of a data analysis tool or related services.

Strongly Preferred

o Familiarity with markets for data analysis tools for high end mathematical modeling such as physics, quantitative finance, econometrics, and new growth areas such as quantitative biology.

Preferred

o Located in San Francisco Bay Area and can visit our office in Sunnyvale, CA if needed.

This is a part-time, contract position (not W-2).

Please send resume or curriculum vitae with cover letter.

No recruiters or head-hunters.

Our job listing on Craig’s List San Francisco (expires November 2017): https://sfbay.craigslist.org/pen/mar/6330138283.html

(C) 2017 by John F. McGowan, Ph.D.

 

Five Ways to Create and Display Slideshows on a Mac

This article explains five ways to create and display slideshows —  sequences of images — on a Mac (Macintosh personal computer) using the software that comes with the Mac.  These ways are:

  1. Using Option Spacebar to play selected images in the Finder
  2. Opening and Playing selected image files with Preview in Finder
  3. Creating Slideshows with Apple Photos
  4. Creating Slideshows with Apple iMovie
  5. Playing the Slideshow Images in Random Order (Shuffle) using the Desktop & Screen Saver control in System Preferences

This article also discusses how to avoid interruption of the slideshow by either the Mac Screen Saver or Energy Saver/Sleep when using an external display and security and privacy issues for slideshows.

These ways of creating and displaying slideshows were tested in detail on a MacBook Air running Mac OS X version 10.12.6 (macOS Sierra), Apple Photos 2.0 (3161.4.140), Preview Version 9.0 (909.18), iMovie version 10.1.7, and System Preferences Version 14.0 (the Desktop & Screen Saver control is part of System Preferences).

Using Option Spacebar to play selected images in the Finder

Select Image Files in Finder

Then, simply press the Option and Spacebar keys on the keyboard.  This will play the selected images as a slideshow in full screen mode.

Option and Spacebar Keys on Mac Keyboard
Option and Spacebar Keys on Mac Keyboard
Floating Slideshow Control for Option Spacebar

On the floating slideshow control, the left pointing arrow goes to the previous slide, the two vertical bars icon pauses playback (solid right pointing arrow resumes playback), the right pointing arrow advances to the next slide and the four squares icon brings up the “index sheet” view of the slide show which shows thumbnails for each slide on a single page:

Index Sheet in Option Spacebar Slideshow

The Option Spacebar method of displaying a slideshow has the advantage that it is simple, quick, and easily accessible from Finder, but gives minimal control over the slideshow.

Opening and Playing selected image files with Preview in Finder

The Apple Preview utility program has a slideshow capability and can be launched from Finder by selecting images in Finder and right clicking to bring up the menu — select open to open all the selected files.  Image files will open with Preview.

Launch Preview for Multiple Files Selected in Finder

Once Preview opens with all of the selected images (left pane in Preview screenshot below), launch the slideshow by selecting View Slideshow

Launch Slideshow from Menu in Preview

The Preview slideshow has a very simple floating control.  The double arrow pointing left goes to the first slide, the two vertical bars pauses the playback, the double arrow pointing right goes to the last slide, and the X in a circle icon exits the slideshow.

Preview Slideshow Floating Control

 

Creating Slideshows with Apple Photos

Both the Option Spacebar method and the Preview method give very limited control over the slideshow.  There is no control over playback speed, transitions between slides, sound, or other options.  Apple Photos can create slideshows quickly with considerable control over these and other options.  It can also export the slideshow as an MPEG-4 video.

The first step to creating a slideshow using Photos is to select the photos for the slideshow in Photos (in some cases, the images may need to be imported into Photos first)

Selecting Photos in Apple Photos 2.0 for Slideshow

Once the photos are selected in the Apple Photos utility program, select the Create Slideshow menu item:

Create Slideshow using Menu Item in Apple Photos 2.0

Photos will create the slideshow with a default name and prompt the user for a custom name if desired:

Slideshow Creation Popup with Default Name in Apple Photos 2.0

 

Rename Slideshow to Another Scenic Walk in Apple Photos 2.0

A named slideshow icon will be added under Projects in the left side pane.  Clicking on Projects will show a view with the thumbnails for each project (slideshow):

Apple Photos Projects with Thumbnails
Apple Photos Projects with Thumbnails

The thumbnail is generated from the first slide in the slideshow unless that slide image is Hidden in photos, in which case a dummy graphic is used.  The “More Art” project in the screenshot above uses a Hidden image as the first slide.  Because the first slide of a slideshow is frequently used as a thumbnail or otherwise displayed by default, it is prudent to select an innocuous slide for the first slide.

Double click on the thumbnail to open the project (slideshow).

Open Project in Apple Photos 2.0
Open Project in Apple Photos 2.0

Play the slideshow by clicking on the right pointing solid triangle below the main slide view (Play Button Icon).

Photos Slideshow Floating Control
Photos Slideshow Floating Control

The Photos slideshow playback has a floating control.  The volume of the slideshow background music or soundtrack is controlled by an icon in this floating control (slider bar on left side).

Photos enables detailed configuration of the slideshow, unlike the Option Spacebar method or the Preview method.

Configuring Slideshow Timing and Other Options
Configuring Slideshow Timing and Other Options
Selecting Slideshow Music or Sound Track
Selecting Slideshow Music or Sound Track
Selecting Slideshow Themes
Selecting Slideshow Themes

As mentioned, Photos can export a slideshow as a fully self-contained MPEG-4 video with full audio.  Click on the export button in the upper right corner of the project.  Photos supports three video resolutions (standard definition or SD, 720p High Definition, and 1080p High Definition).  Here is a short example slideshow created by exporting an MPEG-4 SD video from Photos:

Creating Slideshows with Apple iMovie

Apple iMovie can create slideshows including a soundtrack with detailed control over the duration of each individual slide, individual transitions between slides, and many other fancy Hollywood style effects.  This is probably more than most users need to do.

Scenic Walk Slideshow Project in iMovie
Scenic Walk Slideshow Project in iMovie

Playing the Slideshow Images in Random Order (Shuffle) using the Desktop & Screen Saver control in System Preferences

Remarkably the Mac does not provide an easy way to play the slides in random (or randomized) order, often referred to as Shuffle, in contrast to Windows and other competitors.  The predecessor program to Apple Photos, iPhoto, used to provide a shuffle option, but “it just works” appears to have been deprecated at Apple.

However, in the spirit of the new improved and even more expensive than before Apple, there is an awkward way to play slides in random order (randomized or shuffle) on the Mac using the Mac screen saver.

Screen Saver with Shuffle Slide Order Checked
Screen Saver with Shuffle Slide Order Checked
Choosing Folder with Slides Show Images in Screen Saver
Choosing Folder with Slides Show Images in Screen Saver
Choosing Slideshow Images from Folder in Screen Saver
Choosing Slideshow Images from Folder in Screen Saver
Choose Slides from Apple Photos in Screen Saver
Choose Slides from Apple Photos in Screen Saver
Choose Photos Album as Slideshow in Screen Saver
Choose Photos Album as Slideshow in Screen Saver

One needs to enable the Hot Corners in the Screen Saver to enable the user to immediately launch the randomized slide show by placing the mouse cursor at one of the Hot Corners.  Doesn’t that “just work?”   🙂

Note that one can quickly launch the Desktop & Screen Saver control by using Spotlight on the Mac.  Press Command Spacebar to open spotlight.  Then enter “Desktop & Screen Saver” and just hit return if the utility comes up as the Top Hit (it usually does).

Launching Desktop and Screen Saver with Spotlight
Launching Desktop and Screen Saver with Spotlight

Security and Privacy

Slideshows, slideshow images, slideshow image file names, slideshow folder and album names can all be serious security and privacy concerns.  Apple Photos has a built in feature to hide sensitive images from casual view.

Right Click on Selected Photos to Hide in Apple Photos 2.0
Right Click on Selected Photos to Hide in Apple Photos 2.0

Apple Photos puts all hidden photos in a special Hidden album.  Hidden images are not displayed in Photos, Memories, and several other standard locations.  They are visible in All Photos.  As noted above, if a slideshow project starts with a hidden image, the thumbnail for the slideshow project will be a dummy graphic rather than a thumbnail derived from the hidden image.

By default, the Hidden Album is displayed in the Albums list.  However, it is possible to hide the Hidden Album as well.

Photos Showing Hidden Album
Photos Showing Hidden Album
Menu Item to Hide the Hidden Photo Album
Menu Item to Hide the Hidden Photo Album

Select the Hide Hidden Photo Album menu item from the pulldown View menu to hide the Hidden Photo Album in Apple Photos.

Photos with Hidden Album Hidden
Photos with Hidden Album Hidden

One might wonder about an interface where a hidden album is not hidden by default.  🙂

As mentioned previously, it is probably prudent to choose an innocuous slide for the first slide in a slideshow wherever possible since the first slide is often either directly displayed or used for the thumbnail in some views.

Folder names and album names tend to hang around in various open dialogs and other GUI components on the Mac, so it is best to select secure privacy-protecting names for folders and albums with slide show images.  Generally avoid personally identifiable information, confidential or proprietary information and other sensitive names.

Interruptions on External Displays

In principle, the various applications that display slideshows on the Mac are supposed to block the screen saver and energy saver features while the slideshow is active.  This usually works, but I have experienced a number of cases with an external display where it unpredictably failed.  Either the screen saver or the display blanking happened in the middle of the slide show after the timeout was reached.

For important slideshow presentations or similar situations it is prudent to disable the usual screen saver and energy saver timeouts or to use a third-party program that simulates activity during the slideshow to prevent the screen saver from activating or the mac going to sleep.

These controls (located in System Preferences on the Mac) can be launched directly by typing “Desktop  & Screen Saver” or “Energy Saver” in Spotlight (type Command Key Spacebar to launch spotlight).

Desktop and Screen Saver in Mac System Preferences can Interrupt Some Slideshows
Desktop and Screen Saver in Mac System Preferences can Interrupt Some Slideshows
Power Adapter Turn Display Off Timeout can Interrupt Some Slideshows on Mac
Power Adapter Turn Display Off Timeout can Interrupt Some Slideshows on Mac
Energy Saver Battery Turn Display Off Timeout can Interrupt Some Slideshows on Mac
Energy Saver Battery Turn Display Off Timeout can Interrupt Some Slideshows on Mac

Third party applications such as AntiSleep can emulate activity on the Mac to prevent the timeouts from the Screen Saver and Energy Saver features.  Note that AntiSleep is just one of many such third-party applications.

Launching AntiSleep from Spotlight on Mac OS X 10.12.6
Launching AntiSleep from Spotlight on Mac OS X 10.12.6

Conclusion

Slideshow support is a weak area on the Mac, especially compared to the built-in slideshow features in Windows Explorer.  Apple has actually downgraded its slideshow support from iPhoto to Photos by removing the built-in shuffle/randomized playback feature.

These five methods to create and display slideshows will be more than adequate for the vast majority of users, although more awkward than possible.  It would be better if one could select a group of images in Finder and then directly set playback speed, transition type, shuffle versus ordered playback, and other options from the right click menu or some other accessible method without going through Apple Photos or the Screen Saver.

(C) 2017 by John F. McGowan, Ph.D.

John F. McGowan, Ph.D. solves problems using mathematics and mathematical software, including developing gesture recognition for touch devices, video compression and speech recognition technologies. He has extensive experience developing software in C, C++, MATLAB, Python, Visual Basic and many other programming languages. He has been a Visiting Scholar at HP Labs developing computer vision algorithms and software for mobile devices. He has worked as a contractor at NASA Ames Research Center involved in the research and development of image and video processing algorithms and technology. He has published articles on the origin and evolution of life, the exploration of Mars (anticipating the discovery of methane on Mars), and cheap access to space. He has a Ph.D. in physics from the University of Illinois at Urbana-Champaign and a B.S. in physics from the California Institute of Technology (Caltech).

Caltech Professors Win Nobel Prize in Physics 2017

Kip Thorne and Barry Barish, both emeritus professors of Physics at my alma mater, the California Institute of Technology (Caltech), are among the three winners of the Nobel Prize for Physics in 2017:

https://www.nobelprize.org/nobel_prizes/physics/laureates/2017/press.html

This was for their work performed at Caltech on the LIGO (Laser Interferometer Gravitational-Wave Observatory) gravity wave detection experiment.

Kip Thorne also has a B.S. (Bachelor of Science) degree from Caltech (1962).

A total of 37 faculty and alumni of Caltech have won 38 Nobel Prizes.   Ten (10) of these including Kip Thorne have a B.S. from Caltech.  🙂

(C) 2017 by John F. McGowan, Ph.D.

About

John F. McGowan, Ph.D. solves problems using mathematics and mathematical software, including developing gesture recognition for touch devices, video compression and speech recognition technologies. He has extensive experience developing software in C, C++, MATLAB, Python, Visual Basic and many other programming languages. He has been a Visiting Scholar at HP Labs developing computer vision algorithms and software for mobile devices. He has worked as a contractor at NASA Ames Research Center involved in the research and development of image and video processing algorithms and technology. He has published articles on the origin and evolution of life, the exploration of Mars (anticipating the discovery of methane on Mars), and cheap access to space. He has a Ph.D. in physics from the University of Illinois at Urbana-Champaign and a B.S. in physics from the California Institute of Technology (Caltech).

No, I am not looking for a job at Google!

 

I have been contacted a number of times in the last few months by recruiters or what have turned out to be recruiters from Google.  For the record, I am not currently looking for a job and I am specifically not looking for a job at Google.  🙁

I am developing tools and algorithms for automating complex data analysis, reducing costs and increasing results.  I am interested in conversations with potential customers and interested parties.  You should have a sincere, genuine interest in my work if you contact me.

(C) 2017 by John F. McGowan, Ph.D.

John F. McGowan, Ph.D. solves problems using mathematics and mathematical software, including developing gesture recognition for touch devices, video compression and speech recognition technologies. He has extensive experience developing software in C, C++, MATLAB, Python, Visual Basic and many other programming languages. He has been a Visiting Scholar at HP Labs developing computer vision algorithms and software for mobile devices. He has worked as a contractor at NASA Ames Research Center involved in the research and development of image and video processing algorithms and technology. He has published articles on the origin and evolution of life, the exploration of Mars (anticipating the discovery of methane on Mars), and cheap access to space. He has a Ph.D. in physics from the University of Illinois at Urbana-Champaign and a B.S. in physics from the California Institute of Technology (Caltech).

STEM Employment Related Articles

Inside the Growing Guest Worker Program Trapping Indian Students in Virtual Servitude

An article in the left-wing Mother Jones magazine on Indian students and the OPT program, using students at the University of Central Missouri as examples.

STEM Worker High Turnover Rates

http://www.businessinsider.com/employee-retention-rate-top-tech-companies-2017-8

An article in Business Insider on the possible high turnover rate of many tech companies.  It does not clearly separate the turnover rate and average duration of employment at a company.  A company that is growing rapidly can have a low turnover rate and a low average duration of employment simply because so many employees are new.  If a company doubles in size in two years, half its’ employees will have no more than two years of employment at the company.

Apple, for example, has been growing and hiring rapidly the last several years.  Many employees are new which will pull down the average employment time.   Having worked at Apple from 2014-2016, I suspect it does have a high turnover rate but it is hard to prove due to the apparent rapid growth of the company.

Alleged Age Discrimination in STEM

http://www.bbc.com/future/story/20170828-the-amazing-fertility-of-the-older-mind

An article from the BBC on the considerable ability of older people to learn new things contrary to a common stereotype.

https://www.computerworld.com/article/3090087/it-careers/google-age-discrimination-lawsuit-may-become-monster.html

An article by Patrick Thibodeau at Computerworld on the Google age discrimination class action lawsuit.

Race and Sex Discrimination in STEM

https://www.theguardian.com/technology/2017/aug/07/silicon-valley-google-diversity-black-women-workers

An article in The Guardian questioning Google and other Silicon Valley employer explanations for the low numbers of some groups in their companies, pointing to the large number and percentage of African Americans employees in software engineering in the Washington DC area — generally at government agencies such as NASA and government contractors.

It should be noted that the DC metro area is about 25 percent African-American whereas California as a whole is about 6.5 percent African-American.  Of course, as the article points out, Google and many other tech companies recruit worldwide.

However, Hispanics with visible American Indian ancestry almost certainly make up over 30 percent of California and the San Francisco Bay Area’s population, a comparable or even larger fraction than African-Americans in the DC metro area.  The US Census claims that 38.9 percent of people in California in 2016 were Hispanic-Latino.  Probably 80 to 90 percent of these have visible American Indian ancestry.

The US Census relies on self-identification for race rather than visible appearance.  Hispanics self-identify as white, mixed race, “other race,” and sometimes American Indian/Native American.  My personal impression is that genuine discrimination tends to follow visible appearance and accent/spoken dialect of English.

Hispanic is not a racial category, including people who are entirely European and indeed Northern European in appearance.  At least in my personal experience, most — not all — Hispanics in leadership and engineering positions at high tech companies like Google are European in appearance.  On its diversity web site, Google claims that 4 percent of its workforce in 2017 are Hispanic.

UPDATE (added September 11, 2017)

“At Google, Employee-Led Effort Finds Men Are Paid More Than Women,” by Daisuke Wakabayashi, New York Times, September 8, 2017

The article discusses an internal Google spreadsheet set up by a now former Google employee with self-reported salary and bonus information from Google employees showing women paid less than men.  There is also discussion of the current Labor Department investigation into disparities in salaries between men and women at Google as well as activist investors pressuring Google to disclose information on the salaries of men and women at Google.

 

Articles Questioning STEM Shortage Claims

http://www.techrepublic.com/article/so-much-for-the-stem-shortage/

Tech industry’s persistent claim of worker shortage may be phony, by Michael Hiltzik, Los Angeles Times, August 1, 2015

An article noting the obvious inconsistency between the many layoff announcements in high tech and the claims of a shortage of STEM workers, often by the same employers.

The Open Office Nightmare

Apple staffers reportedly rebelling against open office plan at new $5 billion HQ

An article claiming discontent over the new open office plans at Apple’s new headquarters — the “Spaceship” — in Cupertino.

(C) 2017 John F. McGowan, Ph.D.

About the author

John F. McGowan, Ph.D. solves problems using mathematics and mathematical software, including developing gesture recognition for touch devices, video compression and speech recognition technologies. He has extensive experience developing software in C, C++, MATLAB, Python, Visual Basic and many other programming languages. He has been a Visiting Scholar at HP Labs developing computer vision algorithms and software for mobile devices. He has worked as a contractor at NASA Ames Research Center involved in the research and development of image and video processing algorithms and technology. He has published articles on the origin and evolution of life, the exploration of Mars (anticipating the discovery of methane on Mars), and cheap access to space. He has a Ph.D. in physics from the University of Illinois at Urbana-Champaign and a B.S. in physics from the California Institute of Technology (Caltech).

 

“Introduction to Automating Complex Data Analysis” Video Published

(C) 2017 by John F. McGowan, Ph.D.

About the author

John F. McGowan, Ph.D. solves problems using mathematics and mathematical software, including developing gesture recognition for touch devices, video compression and speech recognition technologies. He has extensive experience developing software in C, C++, MATLAB, Python, Visual Basic and many other programming languages. He has been a Visiting Scholar at HP Labs developing computer vision algorithms and software for mobile devices. He has worked as a contractor at NASA Ames Research Center involved in the research and development of image and video processing algorithms and technology. He has published articles on the origin and evolution of life, the exploration of Mars (anticipating the discovery of methane on Mars), and cheap access to space. He has a Ph.D. in physics from the University of Illinois at Urbana-Champaign and a B.S. in physics from the California Institute of Technology (Caltech).

 

Video of “Automating Complex Data Analysis” Presentation to the Bay Area SAS Users Group

 

This is an edited video of my presentation on “Automating Complex Data Analysis” to the Bay Area SAS Users Group (BASAS) on August 31, 2017 at Building 42, Genentech in South San Francisco, CA.

The demonstration of the Analyst in a Box prototype starts at 14:10 (14 minutes, 10 seconds). The demo is a video screen capture with high quality audio.

Unfortunately there was some background noise from a party in the adjacent room starting about 12:20 until 14:10 although my voice is understandable.

Updated slides for the presentation are available at: https://goo.gl/Gohw87

You can find out more about the Bay Area SAS Users Group at http://www.basas.com/

Abstract:

Complex data analysis attempts to solve problems with one or more inputs and one or more outputs related by complex mathematical rules, usually a sequence of two or more non-linear functions applied iteratively to the inputs and intermediate computed values. A prominent example is determining the causes and possible treatments for poorly understood diseases such as heart disease, cancer, and autism spectrum disorders where multiple genetic and environmental factors may contribute to the disease and the disease has multiple symptoms and metrics, e.g. blood pressure, heart rate, and heart rate variability.

Another example are macroeconomic models predicting employment levels, inflation, economic growth, foreign exchange rates and other key economic variables for investment decisions, both public and private, from inputs such as government spending, budget deficits, national debt, population growth, immigration, and many other factors.

A third example is speech recognition where a complex non-linear function somehow maps from a simple sequence of audio measurements — the microphone sound pressure levels — to a simple sequence of recognized words: “I’m sorry Dave. I can’t do that.”

State-of-the-art complex data analysis is labor intensive, time consuming, and error prone — requiring highly skilled analysts, often Ph.D.’s or other highly educated professionals, using tools with large libraries of built-in statistical and data analytical methods and tests: SAS, MATLAB, the R statistical programming language and similar tools. Results often take months or even years to produce, are often difficult to reproduce, difficult to present convincingly to non-specialists, difficult to audit for regulatory compliance and investor due diligence, and sometimes simply wrong, especially where the data involves human subjects or human society.

A widely cited report from the McKinsey management consulting firm suggests that the United States may face a shortage of 140,000 to 190,000 such human analysts by 2018: http://www.mckinsey.com/business-functions/digital-mckinsey/our-insights/big-data-the-next-frontier-for-innovation.

This talk discusses the current state-of-the-art in attempts to automate complex data analysis. It discusses widely used tools such as SAS and MATLAB and their current limitations. It discusses what the automation of complex data analysis may look like in the future, possible methods of automating complex data analysis, and problems and pitfalls of automating complex data analysis. The talk will include a demonstration of a prototype system for automating complex data analysis including automated generation of SAS analysis code.

(C) 2017 John F. McGowan, Ph.D.

About the author

John F. McGowan, Ph.D. solves problems using mathematics and mathematical software, including developing gesture recognition for touch devices, video compression and speech recognition technologies. He has extensive experience developing software in C, C++, MATLAB, Python, Visual Basic and many other programming languages. He has been a Visiting Scholar at HP Labs developing computer vision algorithms and software for mobile devices. He has worked as a contractor at NASA Ames Research Center involved in the research and development of image and video processing algorithms and technology. He has published articles on the origin and evolution of life, the exploration of Mars (anticipating the discovery of methane on Mars), and cheap access to space. He has a Ph.D. in physics from the University of Illinois at Urbana-Champaign and a B.S. in physics from the California Institute of Technology (Caltech).

 

Automating Complex Data Analysis Presentation to Bay Area SAS Users Group

I will be giving a presentation (about 30 minutes) to the Bay Area SAS User’s Group (BASAS) this Thursday, August 31, 2017 (12:30 PM – 4 PM) at Genentech in South San Francisco, CA: Automating Complex Data Analysis for Fun, Profit, and the Greater Good.
 
Speaker (John F. McGowan, Ph.D.)
Speaker (John F. McGowan, Ph.D.)
Abstract:

Complex data analysis attempts to solve problems with one or more inputs and one or more outputs related by complex mathematical rules, usually a sequence of two or more non-linear functions applied iteratively to the inputs and intermediate computed values. A prominent example is determining the causes and possible treatments for poorly understood diseases such as heart disease, cancer, and autism spectrum disorders where multiple genetic and environmental factors may contribute to the disease and the disease has multiple symptoms and metrics, e.g. blood pressure, heart rate, and heart rate variability.

Another example are macroeconomic models predicting employment levels, inflation, economic growth, foreign exchange rates and other key economic variables for investment decisions, both public and private, from inputs such as government spending, budget deficits, national debt, population growth, immigration, and many other factors.

A third example is speech recognition where a complex non-linear function somehow maps from a simple sequence of audio measurements — the microphone sound pressure levels — to a simple sequence of recognized words: “I’m sorry Dave. I can’t do that.”

State-of-the-art complex data analysis is labor intensive, time consuming, and error prone — requiring highly skilled analysts, often Ph.D.’s or other highly educated professionals, using tools with large libraries of built-in statistical and data analytical methods and tests: SAS, MATLAB, the R statistical programming language and similar tools. Results often take months or even years to produce, are often difficult to reproduce, difficult to present convincingly to non-specialists, difficult to audit for regulatory compliance and investor due diligence, and sometimes simply wrong, especially where the data involves human subjects or human society.

A widely cited report from the McKinsey management consulting firm suggests that the United States may face a shortage of 140,000 to 190,000 such human analysts by 2018: http://www.mckinsey.com/business-functions/digital-mckinsey/our-insights/big-data-the-next-frontier-for-innovation.

This talk discusses the current state-of-the-art in attempts to automate complex data analysis. It discusses widely used tools such as SAS and MATLAB and their current limitations. It discusses what the automation of complex data analysis may look like in the future, possible methods of automating complex data analysis, and problems and pitfalls of automating complex data analysis. The talk will include a demonstration of a prototype system for automating complex data analysis including automated generation of SAS analysis code.

(C) 2017 John F. McGowan, Ph.D.

About the author

John F. McGowan, Ph.D. solves problems using mathematics and mathematical software, including developing gesture recognition for touch devices, video compression and speech recognition technologies. He has extensive experience developing software in C, C++, MATLAB, Python, Visual Basic and many other programming languages. He has been a Visiting Scholar at HP Labs developing computer vision algorithms and software for mobile devices. He has worked as a contractor at NASA Ames Research Center involved in the research and development of image and video processing algorithms and technology. He has published articles on the origin and evolution of life, the exploration of Mars (anticipating the discovery of methane on Mars), and cheap access to space. He has a Ph.D. in physics from the University of Illinois at Urbana-Champaign and a B.S. in physics from the California Institute of Technology (Caltech).