Skip to main content
Springer Nature Link
Log in

Measuring the Efficiency of the Intraday Forex Market with a Universal Data Compression Algorithm

  • Published:
Computational Economics Aims and scope Submit manuscript

Abstract

Universal compression algorithms can detect recurring patterns in any type of temporal data—including financial data—for the purpose of compression. The universal algorithms actually find a model of the data that can be used for either compression or prediction. We present a universal Variable Order Markov (VOM) model and use it to test the weak form of the Efficient Market Hypothesis (EMH). The EMH is tested for 12 pairs of international intra-day currency exchange rates for one year series of 1, 5, 10, 15, 20, 25 and 30 min. Statistically significant compression is detected in all the time-series and the high frequency series are also predictable above random. However, the predictability of the model is not sufficient to generate a profitable trading strategy, thus, Forex market turns out to be efficient, at least most of the time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
€34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

References

  • Alon-Brimer, Y. (2002). Measuring the efficiency of the Israeli stock market by the context tree model. M.Sc. dissertation (in Hebrew), Ben-Gurion University, Israel.

  • Annette, M. G. (1997). Kappa statistics for multiple raters using categorical classifications. In Proceedings of the Twenty-Second Annual SAS Users Group International Conference (available online), San Diego, CA, March.

  • Baetaens D.J.E., van den Berg W.M., Vaudrey H. (1996) Market inefficiencies, technical trading and neural networks. In: Dunis C. (eds) Forecasting financial markets, financial economics and quantitative analysis. Wiley, Chichester, pp 245–260

    Google Scholar 

  • Bahmani-Oskooee, M., Kutan, A. M., & Zhou, S. (2006). Do real exchange rates follow a non-linear mean reverting process in developing countries. EMG Working Paper Series, WP-EMG-02-2006.

  • Begleiter R., El-Yaniv R., Yona G. (2004) On prediction using variable order Markov models. Journal of Artificial Intelligence 22: 385–421

    Google Scholar 

  • Bejerano G., Yona G. (2001) Variations on probabilistic suffix trees: Statistical modeling and prediction of protein families. Bioinformatics 17: 23–43 doi:10.1093/bioinformatics/17.1.23

    Article  Google Scholar 

  • Bellgard, C. D. (2002). Diffusion of forecasting technology: Nonlinearity and the information advantage effect. Ph.D. dissertation, The University of Western Australia.

  • Ben-Gal I., Morag G., Shmilovici A. (2003) CSPC: A monitoring procedure for state dependent processes. Technometrics 45(4): 293–311 doi:10.1198/004017003000000122

    Article  Google Scholar 

  • Ben-Gal I., Shani A., Gohr A., Grau J., Arbiv S., Shmilovici A. et al (2005) Identification of transcription factor binding sites with variable-order Bayesian networks. Bioinformatics 21(11): 2657–2666 doi:10.1093/bioinformatics/bti410

    Article  Google Scholar 

  • Boero G., Marrocu E. (2002) The performance of non linear exchange rate models: A forecasting comparison. Journal of Forecasting, 21: 513–542 doi:10.1002/for.837

    Article  Google Scholar 

  • Buhlmann P., Wyner A.J. (1999) Variable length Markov chains. Annals of Statistics 27(2): 480–513 doi:10.1214/aos/1018031204

    Article  Google Scholar 

  • Chen S.-H., Tan C.-W. (1996) Measuring randomness by Rissanen’s stochastic complexity: Applications to the financial data. In: Dowe D.L., Korb K.B., Oliver J.J. (eds) Information, statistics and induction in science. Singapore, World Scientific

    Google Scholar 

  • Chen, S.-H., & Tan, C.-W. (1999). Estimating the complexity function of financial time series: An estimation based on predictive stochastic complexity. In Proceedings of the 5th International Conference of the Society for Computational Economics, Boston, June 24–26. Also published in Journal of Management and Economics, 3(3) (an electronic journal).

  • Choong, C. K., Poon, W. C., Habibullah, M. S., & Yusop, Z. (2003). The validity of PPP theory in asean five: Another look on cointegration and panel data analysis. Faculty of Accountancy and Management, University Tunku Abdul Rahman.

  • Chung J., Hong Y. (2007) Model-free evaluation of directional predictability in foreign exchange markets. Journal of Applied Econometrics 22: 855–889 doi:10.1002/jae.965

    Article  Google Scholar 

  • Cover T.M. (1991) Universal portfolios. Mathematical Finance 1(1): 1–29 doi:10.1111/j.1467-9965.1991.tb00002.x

    Article  Google Scholar 

  • Cover T.M., Thomas J.A. (1991) Elements of information theory. Wiley, New-York

    Google Scholar 

  • Dacorogna M., Gencay R., Muller U., Olsen R., Pictet O. (2001) An introduction to high-frequency finance. Academic Press, New York

    Google Scholar 

  • Deboeck G. (1994) Trading on the edge: Neural, genetic, and fuzzy systems for chaotic financial markets. Wiley, New York

    Google Scholar 

  • Dempster M.A.H., Payne T.W., Romahi Y., Thompson G.W.P. (2001) Computational learning techniques for intraday FX trading using popular technical indicators. IEEE Transactions on Neural Networks 12(4): 744–754 doi:10.1109/72.935088

    Article  Google Scholar 

  • Fama E.F. (1991) Efficient capital markets: II. The Journal of Finance 46: 1575–1611 doi:10.2307/2328565

    Article  Google Scholar 

  • Fama E.F. (1998) Market efficiency, long term returns, and behavioral finance. Journal of Financial Economics, 49: 283–306 doi:10.1016/S0304-405X(98)00026-9

    Article  Google Scholar 

  • Feder, M., & Federovski, E. (1999), Universal prediction with finite memory. IT Workshop on Detection, Estimation, Classification and Imaging, Santa Fe, NM, USA, 24–26 February.

  • Feder M., Merhav N. (1994) Relations between entropy and error probability. IEEE Transactions on Information Theory 38(1): 259–266 doi:10.1109/18.272494

    Article  Google Scholar 

  • Feder M., Merhav N., Gutman M. (1992) Universal prediction of individual sequences. IEEE Transactions on Information Theory 38(4): 1258–1270 doi:10.1109/18.144706

    Article  Google Scholar 

  • Federovsky E., Feder M., Weiss S. (1998) Branch prediction based on universal data compression algorithm. ACM SIGARCH Computer Architecture News 26(3): 62–72

    Article  Google Scholar 

  • Goldschmidt, P., & Bellgrad, C. (1999). Forecasting foreign exchange rates: Random walk hypothesis, linearity and data freqency. Department of Information Management and Marketing, The University of Western Australia.

  • Hellstrom, T., & Holmstrom, K. (1998). Predicting the stock market. Technical report Ima-TOM-1997-07, Umea University, Sweden. http://citeseer.ist.psu.edu/20821.html.

  • Hutter, M. (2001). Convergence and error bounds for universal prediction of nonbinary sequences. In: Proceedings of the 12th European Conference on Machine Learning ECML-2001 (pp. 239–250).

  • Jensen M. (1978) Some anomalous evidence regarding market efficiency. Journal of Financial Economics, 6: 95–101 doi:10.1016/0304-405X(78)90025-9

    Article  Google Scholar 

  • Kaashoek J.F., Van Dijk H.K. (2002) Neural network pruning applied to real exchange rate analysis. Journal of Forecasting 21: 559–577 doi:10.1002/for.835

    Article  Google Scholar 

  • Kahiri, Y. (2004). Measuring the efficiency of the Forex market via the context tree model. M.Sc. dissertation (in Hebrew), Ben-Gurion University, Israel.

  • Kamruzzaman J., Sarker R.A. (2004) ANN based forecasting of foreign currency exchange rates. Neural Information Processing—Letters and Reviews 3(2): 49–58

    Google Scholar 

  • Lebaron B. (1999) Technical trading rule profitability and foreign exchange intervention. Journal of International Economics 49: 125–143

    Article  Google Scholar 

  • Lo A.W. (2007) Efficient markets hypothesis. In: Blume L., Durlauf S. (eds) The new palgrave: A dictionary of economics (2nd ed). Palgrave MacMillan, London

    Google Scholar 

  • Malkiel B.G. (2003) The efficient market hypothesis and its critics. The Journal of Economic Perspectives 17: 59–82 doi:10.1257/089533003321164958

    Article  Google Scholar 

  • Merhav N., Feder M. (1998) Universal prediction. IEEE Transactions on Information Theory IT- 44: 2124–2147 doi:10.1109/18.720534

    Article  Google Scholar 

  • Millman G.J. (1995) Around the world on a trillion dollars a day. Bantam Press, New York

    Google Scholar 

  • Mills, T. (eds) (2002) Forecasting financial markets. Edward Elgar, UK Cheltenham

    Google Scholar 

  • Neely, C. J. (1997). Technical analysis in the foreign exchange market: A Layman’s guide. Federal Reserve Bank of St. Louis.

  • Orlov Y.L., Filippov V.P., Potapov V.N., Kolchanov N.A. (2002) Construction of stochastic context trees for genetic texts. In Silico Biology 2(3): 233–247

    Google Scholar 

  • Papageorgiou, C. P. (1997). High frequency time series analysis and prediction using Markov models. In Proceedings of the conference on Computational Intelligence for Financial Engineering IEEE/IAFE (pp. 182–188), 23–25 March, New York City, NY.

  • Pring M.J. (1991) Technical analysis explained: The successful investor’s guide to spotting investment trends and turning points. McGraw-Hill, New-York

    Google Scholar 

  • Rissanen J. (1983) A universal data compression system. IEEE Transactions on Information Theory, 29(5): 656–664 doi:10.1109/TIT.1983.1056741

    Article  Google Scholar 

  • Rissanen J. (1984) Universal coding, information, prediction, and estimation. IEEE Transactions on Information Theory 30(4): 629–636 doi:10.1109/TIT.1984.1056936

    Article  Google Scholar 

  • Schwert G. (2003) Anomalies and market efficiency (Chap. 15). In: Constantinides G.M., Harris M., Stulz R. (eds) Handbook of economics of finance. Elsevier, Amsterdam, pp 939–960

    Google Scholar 

  • Shmilovici A., Alon-Brimer Y., Hauser S. (2003) Using a stochastic complexity measure to check the efficient market hypothesis. Computational Economics 22(3): 273–284 doi:10.1023/A:1026198216929

    Article  Google Scholar 

  • Shmilovici, A., & Ben-Gal, I. (2006). Assessing the efficient market hypothesis in stock exchange markets via a universal prediction statistic. Unpublished technical report.

  • Shmilovici A., Ben-Gal I. (2007) Using a VOM model for reconstructing potential coding regions in EST sequences. Computational Statistics 22(1): 49–69 doi:10.1007/s00180-007-0021-8

    Google Scholar 

  • Sim J., Wright C.C. (2005) The kappa statistic in reliability studies: Use, interpretation, and sample size requirements. Physical Therapy 85: 257–268

    Google Scholar 

  • Sullivan R., Timmerman A., White H. (1999) Data-snooping, technical trading rule performance, and the bootstrap. The Journal of Finance 54: 1647–1692 doi:10.1111/0022-1082.00163

    Article  Google Scholar 

  • Taylor M.P., Allen H. (1992) The use of technical analysis in the foreign exchange market. Journal of International Money and Finance 11: 304–314 doi:10.1016/0261-5606(92)90048-3

    Article  Google Scholar 

  • Taylor A.M., Taylor M.P. (2004) The purchasing power parity debate. The Journal of Economic Perspectives 18: 135–158 doi:10.1257/0895330042632744

    Google Scholar 

  • Timmermann A., Granger C.W.J. (2004) Efficient market hypothesis and forecasting. International Journal of Forecasting 20(1): 15–27 doi:10.1016/S0169-2070(03)00012-8

    Article  Google Scholar 

  • Tino, P., Schittenkopf, C., & Dorffner, G. (2000). A symbolics dynamics approach to volatility prediction. In Y. S. Abu-Mustafa & B. LeBaron (Eds.), Proceedings of the 6th International Conference on Computational Finance (pp. 137–151), January 6–8 1999. Cambridge, MA: Leonard N. Stern School of Bussiness, MIT Press.

  • Tsay R. (2002) Analysis of financial time series. Wiley, New York

    Google Scholar 

  • Vert J.P. (2001) Adaptive context trees and text clustering. IEEE Transactions on Information Theory 47(5): 1884–1901 doi:10.1109/18.930925

    Article  Google Scholar 

  • Wu, W., & Shafer, G. (2007). Testing lead-Lag effects under game-theoretic efficient market hypotheses. http://www.probabilityandfinance.com/articles/23.pdf.

  • Yu L., Wang S., Lai K.K. et al (2005) Adaptive smoothing neural networks in foreign exchange rate forecasting. In: Sunderametal V.S. (eds) ICCS, LNCS 3516. Springer, New York, pp 523–530

    Google Scholar 

  • Zaidenraise K.O.S., Shmilovici A., Ben-Gal I. (2007) Gene-finding with the VOM model. Journal of Computational Methods in Science and Engineering 7(1): 45–54

    Google Scholar 

  • Zhang X. (1994) Non-linear predictive models for intra-day foreign exchange trading. Intelligent Systems in Accounting, Finance and Management 3: 293–302

    Google Scholar 

  • Ziv J. (2001) A universal prediction lemma and applications to universal data compression and prediction. IEEE Transactions on Information Theory 47(4): 1528–1532 doi:10.1109/18.923732

    Article  Google Scholar 

  • Ziv, J. (2002). An efficient universal prediction algorithm for unknown sources with limited training data. http://www.msri.org/publications/ln/msri/2002/infotheory/ziv/1/.

  • Ziv J., Lempel A. (1978) Compression of individual sequences via variable rate coding. IEEE Transactions on Information Theory 24: 530–536 doi:10.1109/TIT.1978.1055934

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Systems, Ben-Gurion University, P.O. Box 653, Beer-Sheva, 84105, Israel

    Armin Shmilovici

  2. School of Management, Ben-Gurion University, P.O. Box 653, Beer-Sheva, 84105, Israel

    Yoav Kahiri

  3. Department of Industrial Engineering, Tel-Aviv University, Ramat-Aviv, Tel-Aviv, 69978, Israel

    Irad Ben-Gal

  4. ONO Academic College and School of Management, Ben-Gurion University, P.O. Box 653, Beer-Sheva, 84105, Israel

    Shmuel Hauser

Authors
  1. Armin Shmilovici
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Yoav Kahiri
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Irad Ben-Gal
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Shmuel Hauser
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Armin Shmilovici.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Shmilovici, A., Kahiri, Y., Ben-Gal, I. et al. Measuring the Efficiency of the Intraday Forex Market with a Universal Data Compression Algorithm. Comput Econ 33, 131–154 (2009). https://doi.org/10.1007/s10614-008-9153-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10614-008-9153-3

Keywords

JEL Classification

Mathematics Subject Classification (2000)