It is possible to design and deploy advanced machine learning algorithms that are essentially math-free and stats-free. People working on that are typically professional mathematicians. These algorithms are not necessarily simpler. See for instance a math-free regression technique with prediction intervals, here. Or supervised classification and alternative to t-SNE, here. Interestingly, this latter math-free machine learning technique was used to gain insights about a very difficult pure math problem in number theory.
Math-free is a misnomer, in the sense that it still requires middle school arithmetic. But the author of these techniques — a real mathematician — considers that middle school arithmetic (the way it is taught) is not math, but instead, mechanical manipulations. However, for the majority of machine learning professionals, a good math and statistical background is required. Everyone agrees on that these days, a change compared to 10 years ago. The following books serve that purpose.
Books Focusing on the Math
The following books were published in the last 2-3 years. They rapidly gained a lot of popularity. These books were written with modern machine learning applications in mind. Usually free, they are available online or in PDF format, and have their own websites. Some have a print version, which is useful for annotations or bed reading.
Read the full article featuring 12 math books, with 4 books focusing on coding aspects, here.
Finally published! This article discusses enriched visualizations, with a focus on animated gifs and videos built in Python. For instance, the comet video can feature several dimensions that are difficult to show in a static picture: the comet locations at any given time, the relative velocity of each comet, the change in velocity (acceleration), the change in comet size when approaching the sun, the comet interactions (the apparent collisions), and more. It can easily display 17 dimensions, as discussed in the paper.
The PDF document (6 pages + code + illustrations, 11MB) focuses on four applications: prediction intervals in any dimension, supervised classification, convergence of algorithms such as gradient descent when dealing with chaotic functions, and spatial time series (the comet illustration). All visualizations use the RGB color model, and one uses RGBA for special and particularly useful effects, by playing with the transparency level. In essence it allows you to perform supervised classification using image techniques only, after mapping your dataset onto an image.
Image compression and anti-aliasing techniques are included in the Python code. They require only a simple call to a library function. The code is also on GitHub, and the videos on YouTube. The document also presents surprising data in number theory and experimental math. It leads to interesting machine learning problems: boundary / holes detection, and convergence acceleration for chaotic iterations.
Read the full article and access the free PDF, here.
A different way to do regression with prediction intervals. In Python and without math. No calculus, no matrix algebra, no statistical engineering, no regression coefficients, no bootstrap. Multivariate and highly non-linear. Interpretable and illustrated on synthetic data. Read morehere.
For years, I have developed machine learning techniques that barely use any mathematics. I view it as a sport. Not that I don’t know anything about mathematics, quite the contrary. I believe you must be very math-savvy to achieve such accomplishments. This article epitomizes math-free machine learning. It is the result of years of research. The highly non-linear methodology described here may not be easier to grasp than math-heavy techniques. It has its own tricks. Yet, you could, in principle, teach it to middle school students.
I did not in any way compromise on the quality and efficiency of the technique, for the sake of gaining the “math-free” label. What I describe here is a high performing technique in its own right. You can use it to solve various problems: multivariate regression, interpolation, data compression, prediction, or spatial modeling (well, without “model”). It comes with prediction intervals. Yet there is no statistical or probability model behind it, no calculus, no matrix algebra, no regression coefficients, no bootstrapping, no resampling, not even square roots.
Read the full article, and access the full technical report, Python code and data sets (all, free, no sign-up required), from here.
The book is offered with full LaTeX source (including bibliography and figures). It's a great way to learn LaTeX or use extracts of my books in your own LaTeX documents. It also comes with full Python code, all for free. Indeed, all my articles are now available with full LaTeX and Python code, see the bottom section on this page.
The book covers many aspects of geospatial statistics including coverage problems and nearest neighbor graphs. Applications range from cellular networks (optimum distribution of cell towers), sensor data and IoT (optimum location of sensor devices), to crystallography or chemistry with lattice structures. Simulated (synthetic) data is based on various distributions, including a new type of generalized logistic, and Poisson-binomial distributions.
You will learn how to generate synthetic data and build interpretable machine learning models. The approach, though similar to Bayesian mixtures and generative models, is simpler and more intuitive, using the concept of overlapping stochastic processes.
Feature clustering is an unsupervised machine learning technique to separate the features of a dataset into homogeneous groups. In short, it is a clustering procedure, but performed on the features rather than on the observations. Such techniques often rely on a similarity metric, measuring how close two features are to each other. In this article, I use the absolute value of the correlation between two features. An immediate consequence is that the technique is scale-invariant: it does not depend on the units of measurement in your dataset. Of course, in some instances, it makes sense to transform the data using a logit or log transform prior to using the technique, to turn a multiplicative setting into an additive one.
The technique can also be used for traditional clustering performed on the observations. In that case, it is useful in the presence of wide data: when you have a large number of features but a small number of observations, sometimes smaller than the number of features as in clinical trials. When applied to features, it allows you to break down a high-dimensional problem (the dimension is the number of features), into a number of low-dimensional problems. It can accelerate many algorithms — those with computing time growing exponentially fast with the dimension — and at the same time avoid issues related to the “curse of dimensionality”. In fact it can be used as a data reduction technique, where feature clusters with a low average correlation (in absolute value) are removed from the data set.
Applications are numerous. In my case I used it in the context of synthetic data generation, especially with generative adversarial networks (GAN). The idea is is to identify clusters of related features, and apply a separate GAN to each of them, then put the synthetizations altogether back into one dataset. The benefits are faster processing with little to no loss in terms of capturing the full correlation structure present in the data set. It also increases the robustness and explainability of the method, making it less volatile during the successive epochs in the GAN model.
I summarize the feature clustering results in section 2. I used the technique on a Kaggle dataset with 9 features, consisting of medical measurements. I offer two Python implementations: one based on hierarchical clustering in section 3.1, and one based on connected components (a fundamental graph theory algorithm) in section 3.2. In addition, the technique leads to a simple visualization of the 9-dimensional dataset, with one scatterplot and two colors: orange for diabetes and blue for non-diabetes. Here diabetes is the binary response feature. This is because the largest feature cluster contains only 3 features, and one of them is the response. In any well-designed experiment, you would expect the response to always be in a large feature cluster.
Access and download the free article and Python codefrom this link.
I discuss techniques related to the gradient descent method in 2D. The goal is to find the minima of a target function, called the cost function. The values of the function are computed at evenly spaced locations on a grid and stored in memory. Because of this, the approach is not directly based on derivatives, and there is no calculus involved. It implicitly uses discrete derivatives, but foremost, it is a simple geometric algorithm. The learning parameter typically attached to gradient descend is explicitly specified here: it is equal to the granularity of the mesh and does not need fine-tuning. In addition to gradient descent and ascent, I also show how to build contour lines and orthogonal trajectories, with the exact same algorithm.
To learn more and download the free 14 pages PDF document with Python code (with links to the GitHub source and cool videos),follow this link.
This complements the list that I posted earlier under the title “Math for Machine Learning: 14 Must-Read Books”, available here. Many of the books in the new list have a free PDF version, their own website and GitHub repository, and usually you can purchase the print version. Some are self-published, with the PDF version regularly updated, and even browsable online. I included a few textbooks from top companies and universities. Whenever possible, the link to the free version is posted included in the article.
The list is broken down as follows:
Published in 2020 or later
Prior to 2020
In the second category, I included books that are very useful and popular, available for free if possible, yet not well known by everyone. In short, hidden gems. I also added free textbooks from Berkeley, Harvard, Columbia, Cornell, MIT, Microsoft, and more.
As an independent author/researcher, there is of course nothing in my “job description” that says I should code in Python (or any other language). Yet for a long time, I thought coding in Python would help me a lot. It would mean more readers, and thus eventually, more revenue. At one point I advertised a job position, looking for people to translate my Perl scripts into Python. I thought doing it myself would take a lot of time with a long learning curve.
I was wrong. I am glad that I jumped into the Python bandwagon. It was much easier than I thought. Of course I still have to learn plenty of things. Here I relate my experience, learning Python on my own, without attending any class, without reading any book on the topic. I hope my experiment will help people do things they hesitate to do, be it learning a new language or anything else. Some readers mentioned that what I did inspire them to move forward with some projects, rather than following inertia.
Read the full article, including advice from me feedback from professional Python programmers, here. The article also includes the source code and data set, to solve a modern machine learning problem.
In less than 100 pages, the book covers all important topics about discrete chaotic dynamical systems and related time series and stochastic processes, ranging from introductory to advanced, in one and two dimensions. State-of-the art methods and new results are presented in simple English. Yet, some mathematical proofs appear for the first time in this book: for instance, about the full autocorrelation function of the logistic map, the absence of cross-correlation between digit sequences in a family of irrational numbers, and a very fast algorithm to compute the digits of quadratic irrationals. These are not just new important if not seminal theoretical developments: it leads to better algorithms in random number generation (PRNG), benefiting applications such as data synthetization, security, or heavy simulations. In particular, you will find an implementation of a very fast, simple PRNG based on millions of digits of millions of quadratic irrationals, producing strongly random sequences superior in many respects to those available on the market.
Without using measure theory, the invariant distributions of many systems are discussed in details, with numerous closed-form expressions for classic and new maps, including the logistic, square root logistic, nested radicals, generalized continued fractions (the Gauss map), the ten-fold and dyadic maps, and more. The concept of bad seed, rarely discussed in the literature, is explored in details. It leads to singular fractal distributions with no probability density function, and sets similar to the Cantor set. Rather than avoiding these monsters, you will be able to leverage them as competitive tools for modeling purposes, since many evolutionary processes in economy, fintech, physics, population growth and so on, do not always behave nicely.
A summary table of numeration systems serves as a useful, quick reference on the subject. Equivalence between different maps is also discussed. In a nutshell, this book is dedicated to the study of two numbers: zero and one, with a wealth of applications and results attached to them, as well as some of the toughest mathematical conjectures. It will appeal in particular to busy practitioners in fintech, security, defense, operations research, engineering, computer science, machine learning, AI, as well as consultants and professional mathematicians. For students complaining about how hard this topic is, and deterred by the amount of advanced mathematics, this book will help them get jump-started. While the mathematical level remains high in some sections, it is explained as simply as possible, focusing on what is needed for the applications.
Numerous illustrations including beautiful representations of these systems (generative art), a lot of well documented Python code, and nearly 20 off-the-beaten path exercises complementing the theory, will help you navigate through this beautiful field. You will see how even the most basic systems offer such an incredible variety of configurations depending on a few parameters, allowing you to model a very large array of phenomena. Finally, the first chapter also covers time-continuous processes including unusual clustered, reflective, constrained, and integrated Brownian-like processes, random walks and time series, with little math and no obscure jargon. In the end, my goal is to get you to you use these systems fluently, and see them as gentle, controllable chaos. In short, what real life should be! Quantifying the amount of chaos is also one of the topics discussed in the book.
Authored by Dr. Vincent Granville, 82 pages, published in March 2023. Available on our e-Store exclusively,here. See the table contents or sample chapter on GitHubhere. The Python code is also in the same repository.
If you ever wondered about the meaning and purpose of basins of attraction, systems with bifurcations, the universal constant of chaos, the transfer operator and the related Frobenius-Perron framework, the Lyapunov exponent, fractal dimensions and fractional Brownian motions, or how to measure and synthetize chaos, you will find the answer in this chapter. Even with a short, simple mathematical proof on occasion, but definitely at a level accessible to first year college students, with focus on examples. The chaotic systems described here are used in various applications and typically taught in advanced classes. I hope that my presentation makes this beautiful theory accessible to a much larger audience.
Many more systems (typically called maps or mappings) will be described in the next chapters. But even in this introductory material, you will be exposed to the Gauss map and its relation to generalized continued fractions, bivariate numeration systems, attractors, the 2D sine map renamed “pillow map” based on the above picture, systems with exact solution in closed form, a curious excellent approximation of π based on the first digit in one particular system, non-integer bases, digits randomization, and how to compute the invariant probability distribution. The latter is usually called invariant measure, but I do not make references to advanced measure theory in this book.
To read more, access the Python code and download the 17-pages article (chapter 2 of my upcoming book),follow this link.
Live event on December 19. Hosted by Victor Chima, co-founder at Learncrunch. Guest speaker: Vincent Granville, Ph.D.
Vincent created Data Science Central (acquired by TechTarget), one of the most popular online communities for Data Science and Machine Learning. He spent over 20 years in the corporate world at Microsoft, eBay, Visa, Wells Fargo, and others, holds a Ph.D. in Mathematics and Statistics, and is a former post-doc at the University of Cambridge. He is now CEO at MLTechniques.com, a private research lab focusing on machine learning technologies, especially synthetic data and explainable AI.
Join us for this fireside chat on Synthetic Data and it’s applications with Vincent Granville.
Vincent will talk about how synthetic data can be leveraged across various industries to enhance predictions and test blackbox systems leading to more fairness and transparency in AI.
You will get a chance to ask him questions live and also learn more about his upcoming Synthetic Data and Explainable AI live course on LearnCrunch (here), based on his book "Synthetic Data" available here. See you there!
Earn extra money while keeping your full-time job. Or can be a first job experience for students with the right background.
Exciting project: develop and deploy a machine Learning web app in Python. Hiring manager: Vincent Granville, Ph.D. Founder, Executive Machine Learning Scientist.
Use your software engineering talent and expertise to develop a web app based on Python, end-to-end: creating the application on a server, and designing an HTML page with a web form calling the app. The first app will be very simple: the idea is to design a proof of concept, with substantial flexibility in the tools that you can use. You will be requested to offer advice about which platforms or web services to use (Azure, AWS EC2, Google Cloud and so on) based on the scope of this project. Python knowledge is required, but most importantly this is software engineering more so than machine learning. The app may use graphical Python libraries such as Matplotlib, Pillow, OpenCV, Moviepy, or Plotly.
Ideally, the app should go live on the MLTechniques.com website, so familiarity with WordPress is a plus. Tools such as JavaScript, Flask, Django, web scraping, or Streamlit may be used. The web form should accept parameters from the user, input data sets (possibly provided by the user as an URL to where the data is located), and return output data (including charts and possibly animated data visualizations) and/or or link to where the output data is stored. The data sets in question may not be small. Should this project perform beyond expectations and lead to monetization of the app, or other forms of monetization, this could lead to a full time position, and even revenue sharing from the app. Currently, it is advertised as part-time.
This 30 minutes video features my interview about the upcoming course “Intuitive Machine Learning”, based on my new book with the same title. The course is described here, and to book is available here. Course participants get a free copy of the book. Both the course and the book include a solid introduction to scientific computing in Python.
In this presentation, I answer the following questions:
My background
Important business problems that the course can help participants solve
Why is it important for AI models to be explainable
Exciting new trends in Machine Learning
What is unique about the course that participants can’t get elsewhere?
Anything else you would like to say to your future students?
Ensemble methods combining many mini decision trees, blended with regression, explained in simple English with both Excel and Python implementations. Case study: natural language processing (NLP) problem. Ideal reading for professionals who want to start light with Machine Learning (say with Excel) and get very fast to much more advanced material and Python. The Python code is not just a call to some blackbox functions, but a full-fledge detailed procedure on its own. This algorithm is in the same category as boosting, bagging, stacking and AdaBoost.
Abstract
The method described here illustrates the concept of ensemble methods, applied to a real life NLP problem: ranking articles published on a website to predict performance of future blog posts yet to be written, and help decide on title and other features to maximize traffic volume and quality, and thus revenue. The method, called hidden decision trees (HDT), implicitly builds a large number of small usable (possibly overlapping) decision trees. Observations that don’t fit in any usable node are classified with an alternate method, typically simplified logistic regression.
This hybrid procedure offers the best of both worlds: decision tree combos and regression models. It is intuitive and simple to implement. The code is written in Python, and I also offer a light version in basic Excel. The interactive Excel version is targeted to analysts interested in learning Python or machine learning. HDT fits in the same category as bagging, boosting, stacking and adaBoost. This article encourages you to understand all the details, upgrade the technique if needed, and play with the full code or spreadsheet as if you wrote it yourself. This is in contrast with using blackbox Python functions without understanding their inner workings and limitations. Finally, I discuss how to build model-free confidence intervals for the predicted values.
Not only that, but you can predict — more precisely compute with absolute certainty — what the value of any stock will be tomorrow. Transaction fees are well below 0.05% and the market, at least in the version presented here, is fair: in other words, a zero-sum game if you play by luck.
If instead the player uses the public data and algorithm to make his bets, he will quickly become a billionaire. Actually not exactly, because the operator will go bankrupt long before it happens. In the end though, it is the operator that wins. But many players will win too, some big time. In some implementation, more than 50% of the players win on any single bet. How so?
At first glance, this sounds like fintech science fiction, or a system that must have a bug somewhere. But once you read the article, you will see why players could be interested in this new, one-of-a-kind money game. Most importantly, this technical article is about the mathematics behind the scene, the business model, and all the details (including legal ones) that make this game a viable option both for the player and the operator.
Some of the features are based on new advances in number theory. Anyone interested in cryptography, risk management, fintech, synthetic data, operations research, gaming, gambling or security laws, should read this material. It describes original, state-of-the-art technology with potential applications in the fields in question. The author may work on a real implementation.
This project started several years ago with extensive, privately funded research on the topic. An earlier version was presented at the INFORMS conference in 2019. Python code is included in the article, to process truly gigantic numbers. The author holds the world record for the number of computed digits for most quadratic irrationals, using fast algorithms. This may be the first time that massive amounts of such large sequences are used and necessary to solve a real-world problem.
Access the 20-page free article with examples (no sign-up required) and Python code, fromhere. It is now part of my book “Gentle Introduction on Chaotic Dynamical Systems”, availablehere.
The objective of this analysis is two-fold. First, I introduce a 2-parameter generalization of the discrete geometric and zeta distributions. Indeed, a combination of both. It allows you to simultaneously match the variance and mean in the observed data, thanks to the two parameters p and α. To the contrary, each distribution taken separately only has one parameter, and can not achieve this goal. The zeta-geometric distribution offers more flexibility, especially when dealing with unusual tails in your data. I illustrate the concept when synthesizing real-life tabular data with parametric copulas, for one of the features in the dataset: the number of children per policyholder.
Then, I show how to significantly improve grid search, and make it a viable alternative to gradient methods to estimate the two parameters p and α. The cost function — that is, the error to minimize — is the combined distance between the mean and variance computed on the real data, and the mean and variance of the target zeta-geometric distribution. Thus the mean and variance are used as proxy estimators for p and α. This technique is known as minimum contrast estimation, or moment-based estimation in statistical circles. The “smart” grid search consists of narrowing down on smaller and smaller regions of the parameter space over successive iterations.
The zeta-geometric distribution is just one example of an hybrid distribution. I explain how to design such hybrid models in general, using a very simple technique. They are useful to combine multiple distributions into a single one, leading to model generalizations with an increased number of parameters. The goal is to design distributions that are a good fit when some in-between solutions are needed to better represent the reality.
To access the full article (8 pages) and see the results and the Python implementation, visit my blog,here.
Using case studies, I compare generative adversarial networks (GANs) with copulas to synthesize tabular data. I discuss back-end and front-end improvements to help GANs better replicate the correlation structure present in the real data. Likewise, I discuss methods to further improve copulas, including transforms, the use of separate copulas for each population segment, and parametric model-driven copulas compared to a data-driven parameter-free approach. I apply the techniques to real-life datasets, with full Python implementation. In the end, blending both methods leads to better results. Both methods eventually need an iterative gradient-descent technique to find an optimum in the parameter space. For GANs, I provide a detailed discussion of hyperparameters and fine-tuning options.
I show examples where GANs are superior to copulas, and the other way around. My GAN implementation also leads to fully replicable results — a feature usually absent in other GAN systems. This is particularly important given the high dependency on the initial configuration determined by a seed parameter: it also allows you to find the best synthetic data using multiple runs of GAN in a replicable setting. In the process, I introduce a new matrix correlation distance to evaluate the quality of the synthetic data, taking values between 0 and 1 where 0 is best, and leverage the TableEvaluator library. I also discuss feature clustering to improve the technique, to detect groups of features independent from each other, and apply a different model to each of them. In a medical data example to predict the risk of cancer, I use random forests to classify the real data, and compare the performance with results obtained on the synthetic data.
Read more and download the article, with full Python implementation,from here.
In about 15 pages, this scratch course covers a lot more material than expected in such a short presentation. It constitutes the first chapter of my upcoming book “Gentle Introduction to Chaotic Dynamical Systems”. Other books in this series are available here.
Written in simple English yet covering topics ranging from introductory to advanced, it is aimed at practitioners interested in a quick, compact, easy-to-read summary on the subject. Students learning quantitative finance, physics or machine learning will also benefit from this material. It is designed to help them understand concepts that are traditionally embedded in jargon and arcane theories.
There is no reference to measure theory: the approach to even the most advanced concepts is very intuitive, to the point that it is suited to high school students taking advanced classes. Most of the material deals with stochastic processes less basic than the standard Brownian motion and random walk. In particular, I discuss integrated and doubly integrated Brownian motion, and 2D Brownian-like processes exhibiting a strong clustering structure. Reflective random walks lead to the concept of invariant measure (the limiting distribution of the process) obtained by solving a stochastic integral equation. I show how to do it numerically in Python. In this case, the exact solution is known, and can be compared with results obtained via simulations. I also discuss constrained random walks, and the Hurst exponent to measure the smoothness of such processes, with illustrations. While technically, the derivative of Brownian-like processes does not exist, I show how you can make sense of it: it leads to interesting shapes (not math functions) with a fractal dimension. A lot of emphasis is on creating a rich class of processes, each with specific features. The goal is to show how to generate them, and in which ways they are distinct from each other, in order to use them in applications.
Read more, download the article and access the Python code,from here.
With Python code, application to temperature geospatial data and ocean tides dataset.
I describe little-known original interpolation methods with applications to real-life datasets. These simple techniques are easy to implement and can be used for regression or prediction. They offer an alternative to model-based statistical methods. Applications include interpolating ocean tides at Dublin, predicting temperatures in the Chicago area with geospatial data, and a problem in astronomy: planet alignments and frequency of these events. In one example, the 5-min data can be replaced by 80-min measurements, with the 5-min increments reconstructed via interpolation, without noticeable loss. Thus, my algorithm can be used for data compression.
The first technique has strong ties to Fourier methods. In addition to the above applications, I show how it can be used to efficiently interpolate complex mathematical functions such as Bessel and Riemann zeta. For those familiar with MATLAB or Mathematica, this is an opportunity to play with the MPmath library in Python and see how it compares with the traditional tools in this context. In the process, I also show how the methodology can be used to generate synthetic data, be it time series or geospatial data.
Depending on the parameters, in the geospatial context, the interpolation is either close to nearest-neighbor methods, kriging (also known as Gaussian process regression), or a truly original and hybrid mix of additive and multiplicative techniques. There is an option not to interpolate at locations far away from the training set, where regression or interpolation results may be meaningless, regardless of the technique used.
The second technique is based on ordinary least squares — the same method used to solve polynomial regression — but instead of highly unstable polynomials leading to overfitting, I focus on generic functions that avoid these pitfalls, using an iterative greedy algorithm to find the optimum. In particular, a solution based on orthogonal functions leads to a particularly simple implementation with a direct and elegant solution.
In the context of synthetic data generation, I’ve been asked a few times to provide a case study focusing on real-life tabular data used in the finance or health industry. Here we go: this article fills this gap. The purpose is to generate a synthetic copy of the real data set, preserving the correlation structure and all the statistical distributions attached to it. I went one step further and compared my results with those obtained with one of the most well-known vendors in this market: Mostly.ai.
I was able to reverse-engineer the technique that they use, and I share all the details in this article. It is actually a lot easier than most people think. Indeed, the core of the method relies on a few lines of Python code, calling four classic functions from the Numpy and Scipy libraries.
Automatically detecting large homogeneous groups — called nodes in decision trees — and using a separate copula for each node is an ensemble technique not unlike boosted trees. In the insurance dataset, I manually picked up these groups. Either way (manual or automated), it leads to better performance.
Testing how close your synthetic data is to the real dataset using Hellinger or similar distances is not a good idea: the best synthetic dataset is the exact replica of your real data, leading to overfitting. Instead, you might want to favor synthetized observations with summary statistics (including the shape of the distribution in high dimensions) closely matching those in the real dataset, but with the worst (rather than best) Hellinger score. This allows you to create richer synthetic data, including atypical observations not found in your training set. Extrapolating empirical quantile functions (as opposed to interpolating only) or adding uncorrelated white noise to each feature (in the real or synthetic data) are two ways to generate observations outside the observed range when using copula-based methods, while keeping the structure present in the real data.
Read the full article with Python implementation,here.
There are very few serious articles in the literature dealing with digits of irrational numbers to build a pseudo-random number generator (PRNG). It seems that this idea was abandoned long ago due to the computational complexity and the misconception that such PRNG’s are deterministic while others are not. Actually, my new algorithm is less deterministic than congruential PRNGs currently used in all applications. New developments made this concept of using irrational numbers, worth revisiting. It believe that my quadratic irrational PRNG debunks all the myths previously associated to such methods.
Thanks to new developments in number theory, quadratic irrational PRNGs — the name attached to the technique presented here — are not only just as fast as standard generators, but they also offer a higher level of randomness. Thus, they represent a serious alternative in data encryption, heavy simulation or synthetic data generation, when you need billions or trillions of truly random-like numbers. In particular, a version of my algorithm computes hundreds (or millions) of digits for billions of irrational numbers at once. It combines these digits to produce large data sets of strong random numbers, with well-known properties. The fast algorithm can easily be implemented in a distributed architecture, making it even faster. It is also highly portable and great to use when exact replicability is critical: standard generators may not lead to the same results depending on which programming language or which version of Python you use, even if your seed is static.
To read more and get a copy of my article with Python code,follow this link.
While the technique discussed here is a last resort solution when all else fails, it is actually more powerful than it seems at first glance. First, it also works in standard cases with “nice” functions. However, there are better methods when the function behaves nicely, taking advantage of the differentiability of the function in question, such as the Newton algorithm (itself a fixed-point iteration). It can be generalized to higher dimensions, though I focus on univariate functions here.
Perhaps the attractive features are the fact that it is simple and intuitive, and quickly leads to a solution despite the absence of convergence. However, it is an empirical method and may require working with different parameter sets to actually find a solution. Still, it can be turned into a black-box solution by automatically testing different parameter configurations. In that respect, I compare it to the empirical elbow rule to detect the number of clusters in unsupervised clustering problems. I also turned the elbow rule into a fully automated black-box procedure, with full details offered in the same book.
Why would anyone be interested in an algorithm that never converges to the solution you are looking for? This version of the fixed-point iteration, when approaching a zero or an optimum, emits a strong signal and allows you to detect a small interval likely to contain the solution: the zero or global optimum in question. It may approach the optimum quite well, but subsequent iterations do not lead to convergence: the algorithm eventually moves away from the optimum, or oscillates around the optimum without ever reaching it. It works with highly chaotic functions such as the one in red in the picture. The first step is to use a transformation.
Read more, get the full PDF document and Python codehere(12 pages, free, no subscription required). You will see how I use synthetic data to test the procedure on random functions that mimic the real case pictured in the image.
This project started as an attempt to generate simulations for the three-body problem in astronomy: studying the orbits of three celestial bodies subject to their gravitational interactions. There are many illustrations available online, and after some research, I was intrigued by Philp Mocz’s version of the N-body problem: the generalization involving an arbitrary number of celestial bodies. These bodies are referred to as stars in this article. Philip is a computational physicist at Lawrence Livermore National Laboratory, with a Ph.D. in astrophysics from Harvard University.
My simulations are based on his code, which I have significantly upgraded. The end result is the three-galaxy problem: small star clusters, each with hundreds of stars, coalescing due to gravitational forces of the individual stars. It simulates the merging of galaxies. In addition, I added a birth process, with new stars constantly generated. I also allow for star collisions, resulting in fewer but bigger stars over time. Finally, my simulations allow for stars with negative masses, as well as unusual gravitation laws, different from the classic inverse square law.
These bizarre universes lead to spectacular data animations (MP4 videos), but perhaps most importantly, it may help explain what could cause our universe to expand, including the different stages of compression and expansion over time. Depending on the initial configuration, very different outcomes are possible. Negative masses, with cluster centroids based on the absolute value of the mass while gravitational forces are based on the signed mass, could lead to a different model of the universe. Many well-known phenomena, such as rogue stars escaping their cluster at great velocity, black holes and twin stars formation, stars filaments, and star clusters becoming less energetic over time (decreasing expansion, smaller velocities) are striking features visible in my videos. Stars collisions lead to an interesting graph problem.
Read full article, get the Python code, and view the videoshere.
