On generic complexity of decidability problem for diophantine systems in the Skolem's form | Prikladnaya Diskretnaya Matematika - Applied Discrete Mathematics. 2017. № 37. DOI: 10.17223/20710410/37/8

Generic-case approach to algorithmic problems was suggested by Miasnikov, Kapovich, Schupp and Shpilrain in 2003. This approach studies behavior of an algorithm on typical (almost all) inputs and ignores the rest of inputs. This approach has applications in cryptography where it is required that algorithmic problems must be difficult for almost all inputs. Romankov in 2012 shows that the basic encryption functions of many public key cryptographic systems, among which the RSA system and systems, based on the intractability of the discrete logarithm problem, can be written in the language of Diophantine equations. The effective generic decidability of these equations leads to hacking of corresponding systems, therefore it is actual to study the generic complexity of the decidability problem for Diophantine equations in various formulations. For example, Rybalov in 2011 proved that the Hilbert's tenth problem remains undecidable on strongly generic subsets of inputs in the representation of Diophantine equations by so-called arithmetic schemes. In this paper, we study generic complexity of the Hilbert's tenth problem for systems of Diophantine equations in the Skolem's form. We construct generic polynomial algorithm for determination of solvability of such systems over natural numbers (without zero). We prove strongly generic undecidability of this problem for systems over integers and over natural numbers with zero.