2017-12-04 09:44 AM
Hi we have many windows servers in our environment, but for one server I cannot get winrm to work.
1) I have run the winrmconfig.ps script on the server. The output is all BLUE/Green and Yellow. The user has been specified
04/12/2017 14:20:21
CURRENT LISTENER(S) INFORMATION:04/12/2017 14:20:21 Listener: Address = * Transport = HTTPS Port = 5986 Hostname =MYSERVER.MYDOMAIN.COM Enabled = true URLPrefix = wsman CertificateThumbprint = 2e 05 19 d9 45 6f f8 3e 59 7f 23 10 81 fc e7 d16a 70 9d fb ListeningOn = 10.X.X.X, 10.X.X.X, 127.0.0.1, 172.X.X.X, ::1
04/12/2017 14:20:21 SECURITY LOG ACCESS FOR NETWORK SERVICE ACCOUNT CHECK BEGINS(WINRM SERVICE USES THIS ACCOUNT TO READ EVENT LOGS)
04/12/2017 14:20:21 Network Service SID is already added to the Security Channel ACL (Security Analytics can collect Security Event logs using the myuser@MYDOMAIN account)
04/12/2017 14:20:21 SECURITY LOG ACCESS FOR NETWORK SERVICE ACCOUNT CHECK ENDS
04/12/2017 14:20:21
COLLECTION USER RIGHTS CONFIGURATION BEGINS...04/12/2017 14:20:21 Domain: BOE
04/12/2017 14:20:21 Account: MYUSER
04/12/2017 14:20:21 Checking access to the WinRM WMI Plugin (necessary for SID resolution)
04/12/2017 14:20:21 Winrm WMI plugin SDDL:
<PlugInConfiguration xmlns="http://schemas.microsoft.com/wbem/wsman/1/config/PluginConfiguration" Name="WMI Provider" Filename="C:\Windows\system32\WsmWmiPl.dll" SDKVersion="1" XmlRenderingType="text" UseSharedProcess="false" ProcessIdleTimeoutSec="0" RunAsUser="" RunAsPassword="" AutoRestart="false" Enabled="true" OutputBufferingMode="Block"><Resources><Resource ResourceUri="http://schemas.microsoft.com/wbem/wsman/1/wmi" SupportsOptions="true"><Security Uri="" ExactMatch="false" Sddl="O:NSG:BAD:P(A;;GA;;;BA)(A;;GA;;;IU)(A;;GA;;;RM)(A;;GR;;;S-1-5-21-606747145-527237240-6123456789-123346)S:P(AU;FA;GA;;;WD)(AU;SA;GWGX;;;WD)" /><Capability Type="Identify" /><Capability Type="Get" SupportsFragment="true" /><Capability Type="Put" SupportsFragment="true" /><Capability Type="Invoke" /><Capability Type="Create" /><Capability Type="Delete" /><Capability Type="Enumerate" SupportsFiltering="true" /><Capability Type="Subscribe" SupportsFiltering="true" /></Resource><Resource ResourceUri="http://schemas.dmtf.org/wbem/wscim/1/cim-schema" SupportsOptions="true"><Security Uri="" ExactMatch="false" Sddl="O:NSG:BAD:P(A;;GA;;;BA)(A;;GA;;;IU)(A;;GA;;;RM)S:P(AU;FA;GA;;;WD)(AU;SA;GWGX;;;WD)" /><Capability Type="Get" SupportsFragment="true" /><Capability Type="Put" SupportsFragment="true" /><Capability Type="Invoke" /><Capability Type="Create" /><Capability Type="Delete" /><Capability Type="Enumerate" /><Capability Type="Subscribe" SupportsFiltering="true" /></Resource><Resource ResourceUri="http://schemas.dmtf.org/wbem/wscim/1/*" SupportsOptions="true" ExactMatch="true"><Security Uri="" ExactMatch="false" Sddl="O:NSG:BAD:P(A;;GA;;;BA)(A;;GA;;;IU)(A;;GA;;;RM)S:P(AU;FA;GA;;;WD)(AU;SA;GWGX;;;WD)" /><Capability Type="Enumerate" SupportsFiltering="true" /><Capability Type="Subscribe" SupportsFiltering="true" /></Resource><Resource ResourceUri="http://schemas.dmtf.org/wbem/cim-xml/2/cim-schema/2/*" SupportsOptions="true" ExactMatch="true"><Security Uri="" ExactMatch="false" Sddl="O:NSG:BAD:P(A;;GA;;;BA)(A;;GA;;;IU)(A;;GA;;;RM)S:P(AU;FA;GA;;;WD)(AU;SA;GWGX;;;WD)" /><Capability Type="Get" SupportsFragment="false" /><Capability Type="Enumerate" SupportsFiltering="true" /></Resource></Resources><Quotas MaxConcurrentUsers="100" MaxConcurrentOperationsPerUser="100" MaxConcurrentOperations="1500" /></PlugInConfiguration>
04/12/2017 14:20:21 setWmiSDDL looking for SID: S-1-5-21-606747145-527237240-6123456789-123346
04/12/2017 14:20:21 setWmiSDDL SDDL: O:NSG:BAD:P(A;;GA;;;BA)(A;;GA;;;IU)(A;;GA;;;RM)(A;;GR;;;S-1-5-21-606747145-527237240-6123456789-123346)S:P(AU;FA;GA;;;WD)(AU;SA;GWGX;;;WD)
04/12/2017 14:20:21 User myuser@MYDOMAIN with SID S-1-5-21-606747145-527237240-6123456789-123346 is already
added to the WinRM WMI Plugin SDDL (Security analytics can resolve SIDs with this account)04/12/2017 14:20:21 Checking access to the CIM Root (necessary for Event log collection)
04/12/2017 14:20:21 Checking access to the CIM Root for SID S-1-5-21-606747145-527237240-6123456789-123346
04/12/2017 14:20:21 Setup call to GetSecurityDescriptor, parameters:
04/12/2017 14:20:21 Got acl
04/12/2017 14:20:21 Found trustee SID: S-1-5-21-606747145-527237240-6123456789-123346
04/12/2017 14:20:21 User myuser@MYDOMAIN with SID: S-1-5-21-606747145-527237240-6123456789-123346 is already enabled
for WMI access via WinRM (Security Analytics can collect Event logs using this account)04/12/2017 14:20:21 Checking user myuser@MYDOMAIN membership to the Event Log Readers group
04/12/2017 14:20:22 Event Log Readers member: NETWORK SERVICE
04/12/2017 14:20:22 Event Log Readers member: MYUSER
04/12/2017 14:20:22 User myuser@MYDOMAIN is already a member of Event Log Readers group
04/12/2017 14:20:22
COLLECTION USER RIGHTS CHECK ENDS HERE...04/12/2017 14:20:22 OS Version:
2) From the logcollector I can curl to the Server on port 5986 (the https port successfully)
3) I have looked in the steps of the winrm troubleshooting guide and can see that I have kerberos tickets for this service.
Ticket cache: DIR::/var/netwitness/logcollector/runtime/krb5_ccache_dir/tktWWLHCf
Default principal: winrm@MYDOMAIN.COM
Valid starting Expires Service principal
12/04/17 11:48:45 12/04/17 21:48:54 krbtgt/MYDOMAIN.COM@MYDOMAIN.COM
renew until 12/11/17 11:48:45
12/04/17 11:49:25 12/04/17 21:48:54 HTTP/myserver.mydomain.com@MYDOMAIN.COM
12/04/17 11:51:41 12/04/17 21:48:54 host/myserver.mydomain.com@MYDOMAIN.COM
4) I did a packet capture and can see that packets are flowing. But am wondering if there is an issue establishing the TLS connection
5) winrm get/winrm/config seems to be correct
6) I looked in the event log and saw the following
7) setspn -L hostname returns
C:\Windows\system32>setspn -L SERVER
Registered ServicePrincipalNames for CN=SERVER,OU=DEV,OU=Standard,OU=Member Servers 2012,DC=mydomain DC=com:
WSMAN/SERVER
WSMAN/SERVER.mydomain.com
TERMSRV/SERVER.mydomain.com
TERMSRV/SERVER
RestrictedKrbHost/SERVER
HOST/SERVER
RestrictedKrbHost/SERVER.mydomain.com
HOST/SERVER.mydomain.com
😎 The packet capture reveals that there is an encryption alert 21
9) Windows Log is showing:
Does anyone have any ideas?
2017-12-04 04:34 PM
Best thing is to do this:
setspn -Q HTTP/2k8r2-dc1.2k8r2-vcloud*
note the wildcard at the end of the fqdn of the system you have the failure with
this will show any extra spn’s for that fqdn which is most liekly the issue KRB_APP_ERR_MODIFIED typically stems from this issue. The reason for these dup spins is typically because the system hosts some webserver or shrepoint instance that requires a spn due to a requirement to access the web server via a non standard port.
2017-12-05 04:53 AM
Thanks for the help Con
This is the result that I got
Bascially it seem that the command also found som sort of BASPLUGIN111
Do you know if there is anyway around this?
2017-12-05 05:28 AM
Would running
setspn -U -S HTTP/SERVER:5986 logcollectionuser
setspn -U -S HTTP/SERVER.mydomain.com:5986 logcollectionuser
where logcollectionuser is the domain user used to collect logs work?
2017-12-05 09:55 AM
No this didn't seem to work unfortunately.
Looking at:
There is an option in powershell to
EncodePortInServicePrincipalName
This seems to encode the port in the SPN, so when you make a request it knows the SPN you are requesting. I cant see anyway in Linux to make a request that "encodes the port in the SPN"
I'm still requesting HTTP/Servername rather than HTTP/Servername:5986 so that is why it is failing.
HTTP/Servername is being used by another service.
2017-12-05 11:16 AM
the issue with adding the port via linux is complex, there are three layers that sit under the collector , openwsman, curl and kerberos and neither supports ading a port to the spn curently.
the only option right now is to delete that other spn (if you can verify that its not doing something useful).
2017-12-05 11:40 AM
Thank's Con for the help. I cant delete the other SPN as it is part of a Blackberry Enterprise server.
I think we've just hit a Microsoft Brickwall here...
2017-12-05 12:51 PM
I have a case already open with Microsoft, i have to repro this in the VCloud here since the traces they want i doubt anyone will do in production since they want deep traces on the DC and the target system. My thinking is the DC search the ServicePrincipalName is wrong since it is not doing an exact match, of course they will typically either say FAD or we're wrong but we can let the customer chip in too if it comes to it. We now have several instances of this occurring.
2017-12-07 09:29 AM
Not really sure if this is the issue but we have had similar issues and sometimes the configuration on the systems does not have some of the packages required to support the collection method.
Due to the errors on the netwitness side being less than helpful we found that installing the WINRM IIS Services via the server manager has helped in this situation.
2017-12-07 06:41 PM
In this case it's a clash of SPN's
When we ask kerberos for a service tciket for the system we are connecting to we pass a SPN in the form HTTP/fqdn_of_system
In Dave's case someone has assigned HTTP/fqdn_of_system to an actual account (Service Account) so we get a ticket back that is not for the actual System and when we present that to the System to access the WinRM Service it rejects the ST.
It seems that a search for a SPN in AD always seems to prioritize Service Accounts over systems presumably as they are higher in the tree..