Unfortunately there is no direct event on the ScrollViewer to detect change on the scrollbar. You need to get at the children directly. Here's how to detect change on a vertical scrollbar in the ScrollViewer
ScrollBar verticalScrollBar = ((FrameworkElement)VisualTreeHelper.GetChild(viewer, 0)).FindName("VerticalScrollBar") as ScrollBar;
verticalScrollBar.ValueChanged += (s, ev) => button.Visibility = System.Windows.Visibility.Collapsed;
Don't add this code in the constructor, put it in the Loaded event handler of the child contained in the ScrollViewer.