- Simplify the preprocessing (data cleaning, features extraction & selection-> 90% work) & integration into a mining or ML (10%) & http:
//scikit-learn.org (opensource project supported by google & INRIA research group)
- Simplification of data visualization
- Simplification of semi supervised tagging (reduce the tagging/labelling effort)
- Simplification of parameter selection (model included): Hyperopt: A Python library for optimizing machine learning algorithms; SciPy 2013 - YouTube
- For services in the cloud, the biggest show stopper is data transfer (way to slow) & confidentiality.
Saturday, October 19, 2013
Here is a summary of the ML challenges to reach mass market with APIs:
Prior to 2006, neural networks wasn't cool at all and were ignored by the machine learning community (in papers selection), after Bayesian Network domination, SVM introducted by Vapnick and the kernel trick was dominating the literature (kernel engineering boring papers). SVM had a major advantage, its convergence could be proven and it was more interesting for theoretical paper work than gradient descent which is simply a greedy optimization problem which was leading to local minimum problems.
Despite its theoretical amazing capacity (been able to model any functions), no one was able to train big neural networks and it was even worst to train deep one which could represent similar functions with less parameters. Basically, moving beyond shallow
architecture which imply moving back to the non convex problem optimization nightmare.... back to the optimization research painful path.
Despite the community momentum, Lecun, Hinton and Bengio were still doing research on the subject (thinking outside of the box). The deep learning revolution seems to have started from the discovery of Lecun, with its applications of convolution networks to face recognition. The structure of the topology could affect the training capability drastically. How could we learn this embedding? Then Hinton came with its revolution idea of deep belief networks and auto encoder (a generative model) to better initialize parameters and force lower parameters to learn low level features or embedding initialization state, basically adding data driven constraints to the natural manifold inherent to the problem.
Deep learning is not the answer to AI but a major major breakthrough.
Why? Deep learning not just combines unsupervised learning and supervised learning but also solve partially the training of deep architecture which is critical to learn layers of intelligence (beyond shallow architecture). Furthermore, with neural networks, we don't need to make any assumptions about distributions, its scalable and not that hard to implement. Basically, we use unlabeled data to initialize deep architecture and then train it with our label data.
http://www.iro.umontreal.ca/~lisa/pointeurs/TR1312.pdfTo learn more about DeepLearning and available implementation (maintain by the Bengio Lisa Lab), take a look at http://deeplearning.net/tutorial/ and http://fastml.com/pylearn2-in-practice/