The types contained within the My namespace are contained within Microsoft.VisualBasic.dll - they aren't commonly (or ever!) used across other .NET languages. Those within the Application namespace are.
Under the hood, Application.StartupPath does this:
Public ReadOnly Shared Property StartupPath As String
Get
If (Application.startupPath Is Nothing) Then
Dim stringBuilder As System.Text.StringBuilder = New System.Text.StringBuilder(260)
UnsafeNativeMethods.GetModuleFileName(NativeMethods.NullHandleRef, stringBuilder, stringBuilder.Capacity)
Application.startupPath = Path.GetDirectoryName(stringBuilder.ToString())
End If
(New FileIOPermission(FileIOPermissionAccess.PathDiscovery, Application.startupPath)).Demand()
Return Application.startupPath
End Get
End Property
Whilst My.Application.Info.DirectoryPath does this:
Public ReadOnly Property DirectoryPath As String
Get
Return Path.GetDirectoryName(Me.m_Assembly.Location)
End Get
End Property
which calls this:
Public Overrides ReadOnly Property Location As String
<SecuritySafeCritical>
Get
Dim str As String = Nothing
RuntimeAssembly.GetLocation(Me.GetNativeHandle(), JitHelpers.GetStringHandleOnStack(str))
If (str IsNot Nothing) Then
(New FileIOPermission(FileIOPermissionAccess.PathDiscovery, str)).Demand()
End If
Return str
End Get
End Property
GetModuleFileName used in StartupPath is a call to the native Win32 API, and GetLocation used in DirectoryPath involves a "native" call to the .NET CLR Runtime, so you'd need to dig even deeper to find out where it gets its information from.
TL;DR
Use Application.StartupPath as a preference and to help develop good habits, since it doesn't rely on the Microsoft.VisualBasic additions to .NET, and will make the transition to other languages easier should you ever choose to use them.