RecyclerView Accidental Scrolling On Click

When I click on recycler view several times, it scrolls to bottom on random clicks.

I suspect the issue is with emulator. My AVD is Nexus 5 API 27 x86.

Question: How to eliminate this random scrolling ?

Here is minimal example: https://github.com/OleksandrBezhan/RecyclerViewAccidentalScrolling

Activity:

class MainActivity : AppCompatActivity() {

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)
        recycler.layoutManager = LinearLayoutManager(this)
        recycler.adapter = MyAdapter()
    }
}

Adapter:

class MyAdapter : RecyclerView.Adapter<MyAdapter.ViewHolder>() {
    class ViewHolder(view: View, val text: TextView) : RecyclerView.ViewHolder(view)

    override fun onCreateViewHolder(parent: ViewGroup, viewType: Int): ViewHolder {
        val view = LayoutInflater.from(parent.context)
                .inflate(R.layout.item, parent, false)
        return ViewHolder(view, view.text)
    }

    override fun onBindViewHolder(holder: ViewHolder, position: Int) {
        holder.text.text = "Hello world"
    }

    override fun getItemCount() = 3
}

activity_main.xml:

<?xml version="1.0" encoding="utf-8"?>
<android.support.constraint.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".MainActivity">

    <android.support.v7.widget.RecyclerView
        android:id="@+id/recycler"
        android:layout_width="match_parent"
        android:layout_height="match_parent" />

</android.support.constraint.ConstraintLayout>

item.xml:

<?xml version="1.0" encoding="utf-8"?>
<android.support.constraint.ConstraintLayout
    xmlns:android="http://schemas.android.com/apk/res/android" android:layout_width="match_parent"
    android:layout_height="match_parent">

    <TextView
        android:id="@+id/text"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content" />

</android.support.constraint.ConstraintLayout>

Update: I added OnScrollListener and it shows that on accidental scrolls the scroll state goes to SCROLL_STATE_FLING and then SCROLL_STATE_IDLE.

Whereas under normal circumstances the scroll state should go to SCROLL_STATE_TOUCH_SCROLL -> SCROLL_STATE_FLING -> SCROLL_STATE_IDLE.

recycler.addOnScrollListener(object: RecyclerView.OnScrollListener(){
            override fun onScrollStateChanged(recyclerView: RecyclerView?, newState: Int) {
        Log.d("TEST", "onScrollStateChanged: $newState")
    }
})

D/TEST: onScrollStateChanged: 2
D/TEST: onScrollStateChanged: 0 
// 1, 2, 0 in normal scrolling

It seems to be an emulator issue, since there is no problem on real device.