Econometrics

Econometrics is the application of statistical and mathematical methods to competing economic theories. This takes economic models and time-series data sets through statistical trials in order to test, verify and validate hypotheses which contribute towards forecasting future economic trends and predicting economic risk and fiscal shock events. Econometric Data Science methods, widely employed in finance, market risk and fiscal policy analysis, are now being developed based on Bayesian, clustering and wave-form analytics techniques.

A continuous process of research, development and improvement for advancing econometric modelling and economic theory is driving fiscal policy and decision making.  Non-linear econometric techniques are used increasingly in the analysis of cross section and time series observations.  Non-linear Complex and Chaotic Systems, Monte Carlo simulation and Ensemble Member analysis (facilitated by Bayesian / Cluster Analysis / Wave-form techniques) are increasingly being used in economic modelling, risk and fiscal policy analysis.

This process is advancing Econometric Data Science – which now an integral part of economic forecasting and strategic foresight – within a unified. interactive and iterative Economic Modelling framework that orchestrates the complex tasks of data loading and model initiation, model tuning and history matching economic forecasting and predictive analytics. The application of non-linear techniques to econometric problems have been given new impetus from recent advances in computational power and computer techniques. New econometric techniques – Wave-form Analytics, Monte Carlo simulation, Cluster Analysis and Bayesian statistical analysis – are being developed and deployed, extensively advancing econometric modelling knowledge in the areas of macro-economics and finance.

These novel and emerging developments in Econometric Data Science (Bayesian, clustering and wave-form algorithms) combined with advances in Computing Technology such as “Big Data” (Hadoop – aggregation of time-series and cross-sectional datasets) and “Fast Data” (in-memory computing – GPGPUs, SSDs) are paving the way for “real-time” econometrics – the modelling and analysis of cross section / time series data sets within a unified Economic Modelling framework that orchestrates the complex tasks of data loading and model initiation, model tuning and history matching, economic forecasting and predictive analytics.

Collaboration between institutions and individuals in academia, government and the financial services industry is rapidly improving the understanding of economic theory and econometric modelling and advancing the homogeneity and integration of competing economic theories and models – especially wherever suitable time-series data sets exists to support model evaluation and benchmarking. Major advances have taken place in the analysis of cross sectional data sets by means of semi-parametric and non-parametric techniques.

Competing theories and conflicting models are beginning to receive a much more sympathetic treatment and closer integration. Economic analysis is beginning to resolve elements of heterogeneity and conflict in rival economic models – an essential condition for the development of a “standard economic theory” integrating macro- and micro-economic views within a universally valid “standard economic model and risk framework” for market sentiment, price curve prediction, economic modelling, risk forecasting and fiscal policy evaluation.

PROBLEM / OPPORTUNITY DOMAIN

Monetary Theory is a set of ideas about how fiscal policy – in particular Money Supply and Credit Supply – should be conducted by Governments and Central Banks within their economy.  Monetary theory suggests that alternative monetary policies may benefit nations in different ways depending on their unique set of resources and limitations.

Recent history has demonstrated that our old Monetarist economic models are no longer able to explain real-world events. Econometrics, as a unified discipline, is still relatively young – but has been expanding very rapidly over the past few decades, slowly transforming economic thinking away from classic Monetary Theory in favour of a more realistic Austrian School Economic Theory. Conflicting economic theories and competing econometric models – along with other factors that underlie and influence fiscal policy analysis and evaluation – are now being given a much more sympathetic treatment and satisfactory resolution towards future potential convergence and integration. Even now, attempts are being made to resolve those factors of heterogeneity and conflict in economic modelling – an essential condition for the development of a “standard economic theory” which unifies both macro- and micro-economics within a universally valid “standard economic model”.

ECONOMIC HISTORY SINCE WWII

“Creative Destruction drives Disruptive Change”  
~ Joseph Schumpeter – Economist 

“Creative Destruction” is a term coined by the Austrian Finance Minister and Harvard Economist Joseph Schumpeter in his work entitled “Capitalism, Socialism and Democracy” (1942) to denote a “process of industrial mutation constantly changing the economic structure from within, incessantly destroying the old economy, incessantly creating a new economy”.

“Creative Destruction” occurs with Technology Innovation – when the arrival and adoption of new methods of production creates new industries which effectively kills off older, established industries. An example of this is the introduction of personal computers in the 1980’s. This new industry, led by Bill Gates (Microsoft), Claude Shannon (Intel) and Steve Jobs (Apple) destroyed many mid-range and mainframe computer manufacturers. In doing so, technology entrepreneurs created what was to become one of the most important industry sectors of the 20th century.

“Creative Destruction” underpins Austrian School Capital Theory – and occurs when the arrival and adoption of new methods of production effectively kills off older, established industries as capital flows from old to new industries in economies. One factor of Creative Destruction is Disruptive Change – for example, the migration of mature industries such as consumer electronics and textiles – with their associated industrial capacity and employment opportunities – from high-cost / high tariff Western Economies to emerging nations such as Brazil, the former Russian Federation, India and China (the BRICs) in the post-war period. Disruptive Change acts by attracting capital investment away from traditional agriculture, fishing and forestry industries in developing nations to these new industries – and at the same time, industrial capacity and employment opportunities are lost forever from developed Western economies.

A fundamental economic principle underpinning Capital Theory is Disruptive Change through Global Free Trade. One of the “unforeseen consequences” of the four fundamental economic pillars of the freedom of movement of Economic Resources (Capital, People, Goods and Ideas) is Global Free Trade ( Globalisation) – a world-wide economic phenomenon which is another factor of Disruptive Change. Globalisation emerged from the growth in free-market capitalism which proliferated between Economic Entities and Trade Blocks in the post-war period such as: –

• European Free Trade Area (EFTA),
• European Union (EU)
• North American Free Trade Agreement (NAFTA)
• Free Trade Area of the Americas (FTAA)
• Central American Free Trade Agreement (CAFTA)
• Trans-Pacific Partnership (TPP)

Since WWII, Globalisation, which depends upon the cross-border freedom for the unrestricted movement within Trade of Capital – Men, Money, Machines and Materials – has evolved under these various free trade market agreements

As a result of Global Free Trade, the process of free movement of economic resources (Men, Money, Machines and Materials), Globalisation has underpinned unprecedented growth in world trade volume in the post-war economy. This has also driven the re-location of manufacturing industries – with their associated employment opportunities – from high-cost to low-cost economies in emerging industrial countries, introducing new industrial capacity in sectors such as Agronomy, Forestry, Fishing, Mining, Steel Production and the manufacture of Consumer Electronics and Textiles – accompanied by the consequential loss of manufacturing capacity and job opportunities in the west.

Another fundamental economic principle underpinning Capital Theory is Disruptive Change through Technology Innovation. An example of Technology Innovation is the introduction of personal computers in the 1980’s. This new industry, led by Bill Gates (Microsoft), Steve Jobs (Apple) and Claude Shannon (Intel), destroyed numerous mainframe computer manufacturers. In so doing, technology entrepreneurs created a brand new technology paradigm – Personal Computing. The Personal Computing ecosystem (Hardware, Software and Services) became one of the most important and creative economic ecosystems to dominate society in the late 20th century.

Personal computers are now in turn being replaced by mobile devices – smart apps on phones and tablets – a new technology disruptive ecosystem from agile and innovative companies such as Apple, Samsung and Sony. Microsoft and Nokia are in turn being destroyed; Windows-based smart phones and tablets from Microsoft and Nokia now have less than 3% market share – whilst Blackberry clings on to less than 1% of market share.

The economic process of “Creative Destruction” drives global economic growth and development through Disruptive Change – a result of Technology Innovation interacting with cross-border free market trade – Globalisation. “Disruptive Futurism” is derived from analysis and interpretation of Econometrics – Economic Theory and Modelling. The emerging research area of “Disruptive Futurism” is a relatively new academic subject located in the domain of Strategic Foresight – which is a fusion of methods and techniques derived from the disciplines of Business Strategy and Futures Studies.

Disruptive Futurists seek to discover, analyse and interpret the “gales of creative destruction” which were forecast by Austrian economist Joseph Schumpeter in the 1940s – which are blowing harder today than ever before – with unforeseen dangers. The twin disruptive forces of technology-driven innovation coupled with a dynamic global economy are giving birth to emerging digital markets which generate new business models and revenue streams derived from novel products and services.

The Personal Computing Ecosystem (Personal Hardware, Software and Services), which eroded the Mainframe Computing Ecosystem (Enterprise Hardware, Software and Services) is now in turn being challenged by the SMAC Ecosystem (Social Media, Mobile Platforms, Big Data Analytics and Cloud Services) running on Smart Devices from agile and innovative companies such as Apple, Samsung and Sony. As a result, Blackberry, Microsoft and Nokia are, in their turn, being destroyed – Windows-based smart phones and tablets from Microsoft and Nokia hover at around 3% of the Smart Device market, whilst Blackberry clings on to less than 1% of market share.

Understanding Technology Convergence is the Key to Disruptive Futurism

Were you aware that many common and familiar objects in use today exist only as a result of technology convergence? The familiar, everyday passenger vehicle or laptop computer is the culmination of a series of technology consolidation and integration events from a large number of apparently unrelated technological innovations and advancements. Light-weight batteries were developed to provide independence from fixed power sockets and hard-disk drives were made compact enough to be installed in portable devices. The smart phone and tablet resulted from a further convergence of technologies such as cellular telecommunications, mobile internet, and Smart Apps – mini-applications that do not need to run installed on an on-board hard-disk drive.

Clustered Occurrences of co-impacting Events

Nothing in the galaxy, in our world, or in our own personal existence, ever happens in isolation of other places, objects, individuals and events. This is a very simple and fundamental fact about life, nature and the universe. In the same moment as one event occurs or happens in one location, infinite other events are simultaneously taking place in countless other locations, which in turn impact on an innumerable collection of other co-impacted objects, individuals and events.

In order to study and prepare for the future – we need to bear in mind the fact that all objects and events are potentially connected in some way or other. None of these random events occurs in isolation, none are entirely independent or unconnected – as every object in the Universe exerts some influence over every other object – however tenuously. This phenomenon of Event Clustering is something that, through our own experience, we are all familiar with, know about, and can easily follow and understand.

Every Disruptive Futurist seeks to discover a combination, sequence or chain of apparently Random events which occur together in clusters – which together demonstrate transient or instantiated dependencies (are interacting or co-related- that is, they are impacting upon each other in some way or another). Basically, we seek to apply Complex Systems and Chaos Theory thinking to resolving Future Domain problems or opportunities, threats, issues and challenges…..

What factors or forces do we need to consider as being in-scope and critical to the behaviour of Complex Systems? Which other factors or forces have we ignored, overlooked or not considered? What further unknown factors or unseen forces are there which we have not detected – but may still exist – which in turn are impacting on system elements and thus influencing the behaviour of the system…..?

Such are the questions that need to be asked, in order to help us peer into the mists of time and attempt to foresee those possible, probable and likely future events which lie ahead. For example, in a future when existing mobile Smart Apps such as Skype (which is used for person-to-person communication over Fixed and Mobile Broadband in order to avoid both the need and cost of a conventional telephone network connection) – become available not just on Smart Phones – but also on Smart Clothes (wearable computers), or deployed via a man-machine interface, – and is coupled with the further introduction of unlimited and reliable high-speed internet – then the billion dollar smart phone industry will be disrupted, as the Smart Phone becomes rendered obsolete along with its’ ecosystem of Smart Device manufacturers, Smart Apps developers, and Mobile Service Providers.

Disruption through convergence

The example above seems to be a simple, even obvious, statement. However, the disruptive effect of the smart phone on society is a result of technology convergence, which you can see, if you read the statement carefully. The disruption is the convergence of wearable technology, smart applications that aid communication and improved internet access. The three technologies need to advance and come together in convergence to disrupt the smart phone.

Fundamentally, convergence is exactly what the future is about. Therefore, the key ingredient in strategic foresight, logically, is to study current research and development, events, situations, conditions, changes, evolutions and progress in different areas & aspects, in parallel, concurrently. While studying these, what needs to be then done, is to identify what could have a domino or ripple effect across the parallel lines, what could converge and how, at which point. Once this is done effectively, you have effective foresight that can be of commercial and strategic value that can be used for anticipatory planning and strategizing.

The strategic value of understanding convergence

Failed forecasts, foresight or business plans & strategies can be attributed to the lack of understanding of the dynamics of convergence effects. For an example, when manufacturers falter in pricing strategies. The failure could be a result of inadequately factoring the likely convergence of concurrent market developments, such as changing buyer behaviours, alternatives being sold by market competitors and R&D amongst organisations upstream that would allow a supply of parts to market competitors at lower costs.

Strategic Foresight is the Key to Effective Decision Making

In a fast changing world, fiscal policy and strategic decision making is a key factor in tackling social challenges and economic performance. Disruptive events, such as the financial shock waves, human pandemics, resource security issues (food energy and water crisis – FEW) and geopolitical stability interference events – can rapidly change the assumptions upon which planning has been constructed. Such disruptive events are regularly anticipated and stakeholders regularly prepare responses to them but up to now this process has rarely if ever been engaged with in advance of events, leaving policy responses to be largely reactive and constrained by policy decisions already taken.

The long fruition timescales often involved in research and innovation – from initial transformational technology development into economic and social breakthroughs – mean that few decisions which involve critical assumptions about future cycles, patterns, trends or events – are neither made explicit nor publicly tested. Strategic Foresight aims at systematically and transparently connecting the anticipation of uncertain future developments in society, economy, science and technology (including their consequent opportunities and threats) to policy strategy development and implementation. It allows enhancement of the forward-looking qualities of decision-making, but this approach can only be as effective as the strategy processes in which it is embedded. Therefore, it is first of all imperative to establish a systematic and open process of developing strategy, and to conduct it on a regular basis (“rolling process”). Foresight is thus ‘strategic’ in the sense that it is geared towards feeding systematically into strategy development processes.

SOCIAL AND ECONOMIC DEPRIVATION – TROUBLED FAMILIES

The paradox of Disruptive Change – the interaction of Technology Innovation with free market Globalisation – is that Society cannot reap the economic benefits or the rewards of “Creative Destruction” without accepting that there will be both winners and losers in the process of economic transformation. Some individuals will prosper as new economic opportunities are created. Some individuals will become worse off – not just in the short term, but perhaps for the remainder of their lives.

The key enabler for global economic transformation – Globalisation – is the freedom of movement of People, Ideas, Goods and Capital. Free movement of resources – Men, Machines, Money and Materials – and is the mechanism which drives the transfer of wealth generated by legacy technology industries (Cash Cows) which flows relentlessly into emerging successor industries (Rising Stars) – leaving behind social and economic desolation across heritage industrial cities and towns.

Troubled Families are typically characterised as having: –

• no adult family members in full-time employment
• children not engaged within the educational system
• family members involved in crime and anti-social behaviour

As a direct result of Disruptive Change many of the former Mining, Steel-making, Heavy Engineering, Manufacturing and Textiles cities and towns across the UK have lost their old heritage technology industries – causing social and economic disruption due to the loss of employment opportunities through Globalisation – the free cross-border movement of economic resources – Men, Money, Machines and Materials.  Globalisation is the fiscal mechanism driving global economic transformation – the transfer of wealth generated by traditional legacy industries (Cash Cows) flowing relentlessly into new successor industries (Rising Stars) – leaving behind social and economic disruption across heritage “rust-belt” industrial cities and towns in communities which have lost their legacy industries and jobs.

Troubled Families living in social and economic deprivation across our abandoned legacy industry cities and towns are characterised by having no adult family member in full-time employment, children who are not in education and family members involved in crime and anti-social behaviour.  Troubled Families almost always have other long-standing problems which can lead to their children repeating the cycle of disadvantage. It is estimated that there are vulnerable children and child protection problems in over a third of Troubled Families. Estimates also suggest that one-in-five young offenders and over half of all children permanently excluded from school in England – all come from Troubled Families.

Without employment opportunities, vulnerable and disadvantaged Troubled Families face a multitude of other issues and problems – including financial deprivation, relationship breakdown and isolation, homelessness, mental and physical health issues, drug and alcohol dependency, delinquency, anti-social and criminal behaviour, self-harming and domestic violence. This makes it incredibly difficult for authorities to intervene strategically in an effort to unravel their multiple inter-dependant economic, social, judicial, educational, medical, behavioural, emotional, welfare & wellbeing challenges in their chaotic lifestyles.

The direct cost of Troubled Families to the public purse is very significant – approximately £9 billion a year – mostly accounted for by tactical firefighting and immediate response over Troubled Family emergency issues and problems. Troubled Families, who represent 10% of the Sink Estate population across the UK, consume up to 80% of local public resources – police, social and healthcare. Troubled Families take up significant capacity in Judicial Systems and Law Enforcement, NHS (Primary Healthcare Trusts) and Local Authority (Social Services, Housing and Education). Furthermore, the direct and indirect costs incurred through disruptive, anti-social and criminal behaviour along with its related consequential losses, disruption and other effects – carries an inordinate effect across the whole of the economy as well as an adverse impact across society in general.

The resources consumed and money expended on reacting to Troubled Families is mostly uncoordinated – providing neither lasting results nor fundamental changes to lives. Most importantly, this waste of human potential and public resources is no longer sustainable and therefore the Government has committed to a renewed drive to deal with Troubled Families. As part of the Troubled Families Programme, the Government has put in place resources to incentivise and encourage local authorities and their partner agencies to grasp the nettle and develop new ways of working with Troubled Families – focusing on lasting change. These approaches are likely to incur additional multi-agency costs and workload, but will result in a paradigm shift in the way in which we engage with Troubled Families in the future – ultimately reducing public costs and improving social outcomes.

This new results-based funding scheme provides an important financial incentive to better manage Troubled Families. The Troubled Families programme ambition is, however, greater than this. Through this engagement we want to learn not only how to challenge the way in which multi-agency services are delivered and promise a better and brighter future for Troubled Families – but also to change their long-term social outcomes trajectory.

ECONOMETRICS

A continuous process of research, development and improvement for advancing econometric modelling and economic theory is driving fiscal policy and decision making. Non-linear econometric techniques are used increasingly in the analysis of cross section and time series observations. Non-linear Complex and Chaotic Systems, Monte Carlo simulation and Ensemble Member analysis (facilitated by Bayesian / Cluster Analysis / Wave-form techniques) are increasingly being used in economic modelling, risk and fiscal policy analysis.

This process is advancing Econometric Data Science – which now an integral part of economic forecasting and strategic foresight – within a unified. interactive and iterative Economic Modelling framework that orchestrates the complex tasks of data loading and model initiation, model tuning and history matching economic forecasting and predictive analytics. The application of non-linear techniques to econometric problems have been given new impetus from recent advances in computational power and computer techniques. New econometric techniques – Wave-form Analytics, Monte Carlo simulation, Cluster Analysis and Bayesian statistical analysis – are being developed and deployed, extensively advancing econometric modelling knowledge in the areas of macro-economics and finance.

These novel and emerging developments in Econometric Data Science (Bayesian, clustering and wave-form algorithms) combined with advances in Computing Technology such as “Big Data” (Hadoop – aggregation of time-series and cross-sectional datasets) and “Fast Data” (in-memory computing – GPGPUs, SSDs) are paving the way for “real-time” econometrics – the modelling and analysis of cross section / time series data sets within a unified Economic Modelling framework that orchestrates the complex tasks of data loading and model initiation, model tuning and history matching, economic forecasting and predictive analytics.

SCHUMPETER ECONOMIC WAVE SERIES

The existence of such fundamental stable characteristic frequencies in large aggregations of time-series economic data sets (“Big Data”) provides us with strong evidence and valuable information about the inherent structure of Business Cycles. The challenge found everywhere in business cycle theory is how to interpret interacting large scale, long period, compound wave-form (polyphonic) temporal data sets which are variable (dynamic) in nature such as the Schumpeter Economic Wave series (Kitchen Inventory cycle – 1.5-3 years, Juglar Business cycle – 7-11 years, Kusnets Technology Innovation cycle – 20-25 years and Kondriatev Infrastructure Investment cycle – 40-50 years) and Shock Wave Types (Money Supply, Commodity Price and Sovereign Debt Default shock waves)

BAYSEAN ALGORITHMS, CLUSTER ANALYSIS and WAVE-FORM ANALYTICS

Exploring Baysean, Clustering and Wave-form algorithms against time-series and cross-section aggregated “Big Data” Datasets is the key to unlocking Cycles, Patterns and Trends in complex (non-linear) systems – Cosmology, Climate and Weather, Economics and Fiscal Policy. Modelling the impact of Random Events (Weak Signals, Wild Cards and Black Swan Events) acting on Human Activity complex systems (Schumpeter Political, Economic, Social, Industrial, Agronomy and Technology Waves) and Natural complex systems (Bond Cycles – Solar, Oceanic and Atmospheric Climate Forcing) is important to forecast possible, probable and alternative future outcomes and predict future events.

CLUSTER ANALYSIS in “BIG DATA” 

Cluster Analysis is a technique, using a variety of different Clustering Algorithms, for exploring very large volumes of transactional or machine generated (automatic) data, social media and internet content – where data items sharing similar values are aggregated together in order to discover previously unknown or concealed data relationships.

Numerous spatial, mechanical, mathematical and statistical clustering algorithms are available, many of which are “admissible”, but no single clustering algorithm used alone, in isolation, is considered to be “optimal”:  

• K-means
• Kernel K-means
• Spatial Analysis
• Nearest neighbour
• Spectral Clustering
• CHAID Analysis / R
• Ranking Algorithms
• Wave-form Analytics
• Fourier Transformation
• Gaussian mixture model
• Latent Dirichlet Allocation

The CHAID Analysis (Chi Square Automatic Interaction Detection) in R is a natural form of numeric analysis that identifies each independent variable to discover implicit data relationships between variables, along with other data outcomes, in and across single / multiple Data Sets – without any explicit prior assumptions as to the number or nature of Cluster Centrums. This model – using automatic determination in order to identify how each dependent variable is related, reveals implicit natural groupings and explores any other previously hidden data outcomes – is used in cases of risk management, planning and forecasting, predictive analytics, economic modelling, market penetration – offers, promotions, campaigns, predicting and interpreting responses and a multitude of other data-driven research problems.

WAVE-FORM ANALYTICS

Wave-form Analytics is characterised as the discovery of periodic sequences of regular, recurring increased and reduced time-series trends resulting in cyclic phases of high and low temporal activity. Wave-form Analytics utilises algorithms such as Fourier Analysis, Wigner Distribution and the Gabor Spectrogram in the study of compound, interacting wave forms to identify, separate and isolate individual Cycles, Patterns and Trends which are hidden within large-scale, aggregated datasets – Big Data. Wave-form Analytics supports many important applications in the decomposition and analysis of multiple compound cycles in subject areas as diverse as Physics, Astronomy, Climatology, Biomedical Data Analytics, Network Analysis and Economics.

WAVE-FORM ANALYTICS in “BIG DATA”  is an analytic method which utilises wave frequency and time symmetry principles which has been “borrowed” from spectral wave frequency analysis in Physics. Trend-cycle decomposition is a critical technique for testing, validating and verifying wave-form structures in the study of complex cyclic phenomena where multiple (compound) dynamic (variable) wave-forms compete in a large array of interacting and inter-dependant cyclic which are systems driven by both deterministic (human actions) and stochastic (random, chaotic) paradigms. The Wigner-Gabor-Qian (WGQ) spectrogram – an analytical tool based on a synthesis by Qian of the Gabor Spectrogram and Wigner Distribution – has an extraordinary capacity to reveal complex cycles hidden within noisy Wave-frequency and dynamic Time-symmetry Datasets. When we apply the WGQ spectrogram to these datasets we are able to demonstrate distinct wave-form decomposition, pattern analysis, trend forecasting and predictive modelling capability.

WAVE-FORM ANALYTICS in ECONOMETRICS supports an integrated study of complex, compound wave forms to identify hidden Cycles, Patterns and Trends in Big Data – typically characterised as periodic sequences of regular, recurring increased-reduced time-series trends, resulting in distinct cyclic phases of high-low periodic activity. The Wigner-Gabor-Qian (WGQ) spectrogram may be introduced for studying pattern recognition in business cycles. The Gabor Spectrogram – which captures periodic components from the time-frequency distribution series discovered within the observed system – is a toolkit in the LabView System marketed by National Instruments. Exploration of the characteristic frequencies found in very large scale time-series Econometric datasets (Big Data) reveals strong evidence and valuable insights into the inherent stable and enduring fundamental wave structure found across time-series datasets – thus demonstrating business cycles.

The stabilised characteristic frequencies of Economic, Business and Market Cycles, Patterns and Trends, which are exposed by the Wigner-Gabor-Qian (WGQ) spectrogram and Hodrick-Prescott (HP) filter, are widely observed in aggregated economic time series datasets.

 

Arrange a Conversation