TDE Initial Phase

August 25, 2019, 9:33 am

≫ Next: Polybase : SQL Server 2019 CTP3.1 - Scale-Out Group Configuration

≪ Previous: “There is insufficient system memory in resource pool 'default' to run this query” ERROR when using large in-memory table variable

May I know TDE is a random read/write process?

I see the IO size is about 256KB.

Any factor can speed up the TDE initial encryption ?

↧

Polybase : SQL Server 2019 CTP3.1 - Scale-Out Group Configuration

July 31, 2019, 12:40 am

≫ Next: Polybase SQL Server 2019 : External Table Query Result Issue

≪ Previous: TDE Initial Phase

Currently we are testing SQL Server 2019 Polybase scale-out group feature and stuck with the following issue with the configuration.

Server Details : We have 2 server (1 Head Node and 1 Compute Node, both the servers have same hardware configuration and OS installed which is Windows Server 2016 Standard Edition).

The SQL Server and Polybase Services are configured as per the prerequisite and Retirement mentioned in the link

Here are the snapshots of all the configuration done.

Head Node SQL Instance

Registry Key Values

No Errors were recorded in the in the Polybase Log files.

Compute Node Configuration

Registry Key Values

Issue : Post doing the configuration using "Configure Polybase Scale-out Group" UI, we noticed that compute node is not showing up in the SSMS under Polybase->Scale-out Group.However we can see the head node is added under the compute node ->Polybase - Scaleout Grpup.

We also noticed that the Polybase Engine service start mode is set to Disabled and continue to be running

The Register Values on the compute node (DmsControlChannelHostname) also got updated with Head Node Hostname.

While querying Polybase DMV's Queries , the query on compute node continue to run for never to end.

On further troubleshooting , in the windows event log , the following error is recorded

Log Name: PDW Component Failures
Source: PDW Control Diagnostics
Date: 7/31/2019 1:37:53 PM
Event ID: 0
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: sgsqlpbc01.dbprox.local
Description:
sgsqlpbc01 - mpdwsvc(360):

EngineStartup encountered timeout. EngineServer did not update state within timeout limit.

System.TimeoutException: EngineStartup encountered timeout. EngineServer did not update state within timeout limit.
at Microsoft.SqlServer.DataWarehouse.EngineService.EngineService.StartupCoOrdinator.EngineStartup.Start(EngineTimer startupTimer)
at Microsoft.SqlServer.DataWarehouse.EngineService.EngineService.StartupCoOrdinator.StartDBsAndEngine(EngineTimer startupTimer)
at Microsoft.SqlServer.DataWarehouse.EngineService.EngineService.OnStart(String[] args)

at Microsoft.SqlServer.DataWarehouse.EngineService.EngineService.StartupCoOrdinator.EngineStartup.Start(EngineTimer startupTimer)
at Microsoft.SqlServer.DataWarehouse.EngineService.EngineService.StartupCoOrdinator.StartDBsAndEngine(EngineTimer startupTimer)
at Microsoft.SqlServer.DataWarehouse.EngineService.EngineService.OnStart(String[] args)
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="PDW Control Diagnostics" />
<EventID Qualifiers="0">0</EventID>
<Level>2</Level>
<Task>0</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2019-07-31T05:37:53.269047800Z" />
<EventRecordID>60</EventRecordID>
<Channel>PDW Component Failures</Channel>
<Computer>sgsqlpbc01.dbprox.local</Computer>
<Security />
</System>
<EventData>
<Data>sgsqlpbc01 - mpdwsvc(360):

EngineStartup encountered timeout. EngineServer did not update state within timeout limit.

System.TimeoutException: EngineStartup encountered timeout. EngineServer did not update state within timeout limit.
at Microsoft.SqlServer.DataWarehouse.EngineService.EngineService.StartupCoOrdinator.EngineStartup.Start(EngineTimer startupTimer)
at Microsoft.SqlServer.DataWarehouse.EngineService.EngineService.StartupCoOrdinator.StartDBsAndEngine(EngineTimer startupTimer)
at Microsoft.SqlServer.DataWarehouse.EngineService.EngineService.OnStart(String[] args)

at Microsoft.SqlServer.DataWarehouse.EngineService.EngineService.StartupCoOrdinator.EngineStartup.Start(EngineTimer startupTimer)
at Microsoft.SqlServer.DataWarehouse.EngineService.EngineService.StartupCoOrdinator.StartDBsAndEngine(EngineTimer startupTimer)
at Microsoft.SqlServer.DataWarehouse.EngineService.EngineService.OnStart(String[] args)</Data>
</EventData>
</Event>

Regards

Sufian

↧

Polybase SQL Server 2019 : External Table Query Result Issue

August 22, 2019, 9:32 am

≫ Next: SQL Database Error 924 – Database is already open and can only have one user at a time

≪ Previous: Polybase : SQL Server 2019 CTP3.1 - Scale-Out Group Configuration

We have couple of CSV files on Azure Container Folder and accessing those files with Polybase External Table.

All the files has the same structure, data of each file can be queried when accessed separately.

CREATE EXTERNAL TABLE [dbo].[dbs_ACC_DATA]
(

[Account Number] [varchar](50) NULL,[Client ID] [varchar](100) NULL,[Active?] [varchar] (10) NULL,[Individual or Entity Account] [varchar] (10) NULL)

WITH (DATA_SOURCE = [AzureDS],LOCATION = N'/client_files_dbs/file_1.csv',FILE_FORMAT =[az_storage_files],REJECT_TYPE = VALUE,REJECT_VALUE = 1)

SELECT * FROM dbs_ACC_DATA

We are running into the following issues when accessing all the files in the folder

CREATE EXTERNAL TABLE [dbo].[dbs_ACC_DATA]
(

[Account Number] [varchar](50) NULL,[Client ID] [varchar](100) NULL,[Active?] [varchar] (10) NULL,[Individual or Entity Account] [varchar] (10) NULL)

WITH (DATA_SOURCE = [AzureDS],LOCATION = N'/client_files_dbs/',FILE_FORMAT =[az_storage_files],REJECT_TYPE = VALUE,REJECT_VALUE = 1)

Error :

Cannot execute the query "Remote Query" against OLE DB provider "MSOLEDBSQL" for linked server "(null)". 107090;Query aborted-- the maximum reject threshold (1 rows) was reached while reading from an external source: 2 rows rejected out of total 2 rows processed.

Error log from DWEngine_server log file.

CREATE EXTERNAL TABLE [dbo].[dbs_ACC_DATA]
(
	[Account Number] [varchar](50) NULL,	[Client ID] [varchar](100) NULL,	[Active?] [varchar] (10) NULL,	[Individual or Entity Account] [varchar] (10) NULL,	[Single or Joint Account] [varchar] (10) NULL,
	[Number of Account Holders] [varchar] (10) NULL,	[Entity Name] [varchar](300) NULL,	[DBA/Disregarded Entity Name] [varchar](100) NULL,	[Country of Organization/Incorporation] [varchar](2) NULL,
	[First Name] [varchar](300) NULL,	[Middle Name] [varchar](300) NULL,	[Last Name] [varchar](300) NULL,	[Citizenship Country] [varchar](2) NULL,	[Country of Birth] [varchar](2) NULL,	[Date of Birth] [varchar](100) NULL,
	[PR Address - Street 1] [varchar](150) NULL,	[PR Address - Street 2] [varchar](150) NULL,	[PR Address - City] [varchar](100) NULL,	[PR Address - Postal Code] [varchar](50) NULL,	[PR Address - State] [varchar](100) NULL,	[PR Address - Country] [varchar](2) NULL,
	[Mailing Address - Street 1] [varchar](150) NULL,	[Mailing Address - Street 2] [varchar](150) NULL,	[Mailing Address -  City] [varchar](100) NULL,	[Mailing Address - Postal Code] [varchar](50) NULL,	[Mailing Address - State] [varchar](100) NULL,
	[Mailing Address - Country] [varchar](2) NULL,	[Additional Mailing Address Countries] [nvarchar](100) NULL,	[Phone Number] [nvarchar](30) NULL,	[Standing Instructions Country] [varchar](2) NULL,
	[Account Opening Date] [datetime] NULL,	[Classification of a U.S. Person] [int] NULL,	[Foreign TIN Type] [nvarchar](6) NULL,
	[U.S. TIN] [varchar](9) NULL,	[U.S. TIN Type] [nvarchar](6) NULL,	[Registration Code] [nvarchar](100) NULL,	[Email Address] [varchar](200) NULL,	[Title] [varchar](100) NULL,
	[Town of Birth] [varchar](50) NULL,	[Reporting FI Jurisdiction] [varchar](2) NULL,	[Tax Residence] [nvarchar](2) NULL,	[POA Country] [varchar](2) NULL,
	[Hold Mail Country] [varchar](2) NULL,	[CIC Code] [int] NULL,	[FILER1] [varchar](100) NULL,	[FILER2] [varchar](100) NULL,	[FILER3] [varchar](100) NULL,	[FILER4] [varchar](100) NULL,	[FILER5] [varchar](100) NULL,
	[FILER6] [varchar](100) NULL,	[FILER7] [varchar](100) NULL,	[FILER8] [varchar](100) NULL,	[FILER9] [varchar](100) NULL,	[FILER10] [varchar](100) NULL
)
WITH (DATA_SOURCE = [AzureDS],LOCATION = N'/client_files_dbs/',FILE_FORMAT =[az_storage_files],REJECT_TYPE = VALUE,REJECT_VALUE = 1)
 SID333:QID507 [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:05 AM [Thread:3652] [ServerInterface:InformationEvent] (Info, Normal): Starting processor SqlFrontEndWorkProcessor. [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:05 AM [Thread:3652] [ExternalHadoopBridge:InformationEvent] (Info, Normal): Initializing bridge... [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:05 AM [Thread:3652] [JavaBridge:InformationEvent] (Info, Normal): LoadOrAttachJVM: Getting created JVMs if any... [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:05 AM [Thread:3652] [JavaNativeInterface:InformationEvent] (Info, Normal): Attaching JVM to current thread... [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:05 AM [Thread:3652] [ExternalHadoopBridge:InformationEvent] (Info, Normal): Attached to JVM... [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:05 AM [Thread:3652] [HdfsBridgeFileAccess:InformationEvent] (Info, Normal): GetFileMetadata... [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:05 AM [Thread:3652] [JavaBridge:InformationEvent] (Info, Normal): LoadOrAttachJVM: Getting created JVMs if any... [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:05 AM [Thread:3652] [JavaNativeInterface:InformationEvent] (Info, Normal): Attaching JVM to current thread... [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:05 AM [Thread:3652] [ExternalHadoopBridge:InformationEvent] (Info, Normal): Attached to JVM... [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:05 AM [Thread:3652] [HdfsBridgeJniWrapper:InformationEvent] (Info, Normal): Acquired JvmCreationLock... [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:10 AM [Thread:3652] [HdfsBridgeFileAccess:InformationEvent] (Info, Normal): Getting file info for each file in a directory containing total 4 files [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:10 AM [Thread:3652] [HdfsBridgeFileAccess:InformationEvent] (Info, Normal): File info obtained for each file. [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:10 AM [Thread:3652] [ServerInterface:InformationEvent] (Info, Normal): Query SID333:QID507 completed. [Session.SessionId:SID333][Session.IsTransactional:False][Query.QueryId:QID507]
8/23/2019 12:28:10 AM [Thread:4960] [EngineInstrumentation:EngineStopSessionBeginEvent] (Info, Low): Stopping session, in reset connection: False. [Session.SessionId:SID333][Session.IsTransactional:False]
8/23/2019 12:28:10 AM [Thread:4960] [EngineInstrumentation:TdsRemoveConnectionEvent] (Info, Low): Stopping TDS connection: 2609209822992 
8/23/2019 12:28:10 AM [Thread:4556] [TdsRequestProcessor:InformationEvent] (Info, Normal): CancelBatch request started [Session.SessionId:SID333] 
8/23/2019 12:28:10 AM [Thread:4556] [EngineInstrumentation:TdsRemoveConnectionEvent] (Info, Low): Stopping TDS connection: 2609209824864 
8/23/2019 12:28:10 AM [Thread:4556] [EngineInstrumentation:EngineStopSessionBeginEvent] (Info, Low): Stopping session, in reset connection: False. [Session.SessionId:SID333][Session.IsTransactional:False]
8/23/2019 12:28:12 AM [Thread:4556] [PdwTdsNativeToManagedInterop:InformationEvent] (Info, Normal): Incoming TDS connection, Client TDS version: 7 (TDS74). 
8/23/2019 12:28:12 AM [Thread:4556] [EngineInstrumentation:TdsAddConnectionEvent] (Info, Low): Starting TDS connection: 2609209822992 Client: 192.168.10.49:51536, isFromDataSecurityProxy: False, isVnetAddress: False, clientAddressSafe: 192.168.10.49:51536 
8/23/2019 12:28:12 AM [Thread:4556] [DataClassificationConfig:InformationEvent] (Info, Normal): Returning FeatureSwitch DataClassificationCoreEnabled status: False [Session.SessionId:SID334][Session.IsTransactional:False]
8/23/2019 12:28:12 AM [Thread:4556] [EngineInstrumentation:ServerStartSessionEvent] (Info, Low): Started new session, in reset connection: False. [Session.SessionId:SID334][Session.IsTransactional:False]
8/23/2019 12:28:12 AM [Thread:4556] [TdsRequestProcessor:InformationEvent] (Info, Normal): Processing login, authentication scheme: "Ntlm", client application name: "87E0B5F8-7B5A-776E-2D52-AA5CB888E28D" 
8/23/2019 12:28:12 AM [Thread:4556] [PdwTdsNativeToManagedInterop:InformationEvent] (Info, Normal): Data Classification TDS Extension Requested=False, FS Enabled=False, Enabled=False. [Session.SessionId:SID334][Session.IsTransactional:True][Query.QueryId:QID508]
8/23/2019 12:28:12 AM [Thread:1668] [EngineInstrumentation:TdsAddConnectionEvent] (Info, Low): Starting TDS connection: 2609209824864 Client: , isFromDataSecurityProxy: False, isVnetAddress: False, clientAddressSafe:  
8/23/2019 12:28:12 AM [Thread:4556] [EngineInstrumentation:EngineStopSessionBeginEvent] (Info, Low): Stopping session, in reset connection: False. [Session.SessionId:SID334][Session.IsTransactional:False]
8/23/2019 12:28:12 AM [Thread:4556] [EngineInstrumentation:TdsRemoveConnectionEvent] (Info, Low): Stopping TDS connection: 2609209824864 
8/23/2019 12:28:12 AM [Thread:4556] [TdsRequestProcessor:InformationEvent] (Info, Normal): CancelBatch request started [Session.SessionId:SID334] 
8/23/2019 12:28:12 AM [Thread:4636] [EngineInstrumentation:EngineCancelQueryBeginEvent] (Info, Low): Cancel requested. [Session.SessionId:SID334][Session.IsTransactional:False]
8/23/2019 12:28:12 AM [Thread:4556] [EngineInstrumentation:TdsRemoveConnectionEvent] (Info, Low): Stopping TDS connection: 2609209822992 
8/23/2019 12:28:12 AM [Thread:4556] [EngineInstrumentation:EngineStopSessionBeginEvent] (Info, Low): Stopping session, in reset connection: False. [Session.SessionId:SID334][Session.IsTransactional:False]
8/23/2019 12:28:12 AM [Thread:4556] [PdwTdsNativeToManagedInterop:InformationEvent] (Info, Normal): Incoming TDS connection, Client TDS version: 7 (TDS74). 
8/23/2019 12:28:12 AM [Thread:4556] [EngineInstrumentation:TdsAddConnectionEvent] (Info, Low): Starting TDS connection: 2609209822992 Client: 192.168.10.49:51537, isFromDataSecurityProxy: False, isVnetAddress: False, clientAddressSafe: 192.168.10.49:51537 
8/23/2019 12:28:12 AM [Thread:4556] [DataClassificationConfig:InformationEvent] (Info, Normal): Returning FeatureSwitch DataClassificationCoreEnabled status: False [Session.SessionId:SID335][Session.IsTransactional:False]
8/23/2019 12:28:12 AM [Thread:4556] [EngineInstrumentation:ServerStartSessionEvent] (Info, Low): Started new session, in reset connection: False. [Session.SessionId:SID335][Session.IsTransactional:False]
8/23/2019 12:28:12 AM [Thread:4556] [TdsRequestProcessor:InformationEvent] (Info, Normal): Processing login, authentication scheme: "Ntlm", client application name: "87E0B5F8-7B5A-776E-2D52-AA5CB888E28D" 
8/23/2019 12:28:12 AM [Thread:4556] [PdwTdsNativeToManagedInterop:InformationEvent] (Info, Normal): Data Classification TDS Extension Requested=False, FS Enabled=False, Enabled=False. [Session.SessionId:SID335][Session.IsTransactional:True][Query.QueryId:QID509]
8/23/2019 12:28:12 AM [Thread:1668] [EngineInstrumentation:TdsAddConnectionEvent] (Info, Low): Starting TDS connection: 2609209824864 Client: , isFromDataSecurityProxy: False, isVnetAddress: False, clientAddressSafe:  
8/23/2019 12:28:12 AM [Thread:4556] [ServerInterface:InformationEvent] (Info, Normal): Incoming Query:  SID335:QID510 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:12 AM [Thread:3652] [ServerInterface:InformationEvent] (Info, Normal): Starting processor ExecuteMemoProcessor. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [OptimizedStatement:InformationEvent] (Info, Normal): Memo compilation time: 142.5795 ms [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [OptimizedStatement:InformationEvent] (Info, Normal): Sql Server Optimization Clock: 0 s, CPU: 0.002 s. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [OptimizedStatement:InformationEvent] (Info, Normal): SQL Server XML generation Clock : 0.011 s, CPU: 0.01 s [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [ExternalHadoopBridge:InformationEvent] (Info, Normal): Initializing bridge... [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [JavaBridge:InformationEvent] (Info, Normal): LoadOrAttachJVM: Getting created JVMs if any... [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [JavaNativeInterface:InformationEvent] (Info, Normal): Attaching JVM to current thread... [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [ExternalHadoopBridge:InformationEvent] (Info, Normal): Attached to JVM... [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [HdfsBridgeFileAccess:InformationEvent] (Info, Normal): GetFileMetadata... [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [JavaBridge:InformationEvent] (Info, Normal): LoadOrAttachJVM: Getting created JVMs if any... [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [JavaNativeInterface:InformationEvent] (Info, Normal): Attaching JVM to current thread... [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [ExternalHadoopBridge:InformationEvent] (Info, Normal): Attached to JVM... [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:13 AM [Thread:3652] [HdfsBridgeJniWrapper:InformationEvent] (Info, Normal): Acquired JvmCreationLock... [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [HdfsBridgeFileAccess:InformationEvent] (Info, Normal): Getting file info for each file in a directory containing total 4 files [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [HdfsBridgeFileAccess:InformationEvent] (Info, Normal): File info obtained for each file. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [AppConfigPropertyResolver:InformationEvent] (Info, Normal): Configuration property [ReaderHashCost] has been initialized. Source provider: LocalFileSettingsProvider. Value: 0.00021. Default: 0.00021 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [AppConfigPropertyResolver:InformationEvent] (Info, Normal): Configuration property [WriterCost] has been initialized. Source provider: LocalFileSettingsProvider. Value: 2.4E-05. Default: 2.4E-05 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [AppConfigPropertyResolver:InformationEvent] (Info, Normal): Configuration property [InsBulkCpyCost] has been initialized. Source provider: LocalFileSettingsProvider. Value: 0.00024. Default: 0.00024 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [AppConfigPropertyResolver:InformationEvent] (Info, Normal): Configuration property [NetworkCost] has been initialized. Source provider: LocalFileSettingsProvider. Value: 1E-09. Default: 1E-09 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [AppConfigPropertyResolver:InformationEvent] (Info, Normal): Configuration property [ExternalReadCost] has been initialized. Source provider: LocalFileSettingsProvider. Value: 4.8E-05. Default: 4.8E-05 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [AppConfigPropertyResolver:InformationEvent] (Info, Normal): Configuration property [ExternalHadoopWriteCost] has been initialized. Source provider: LocalFileSettingsProvider. Value: 0.00024. Default: 0.00024 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [AppConfigPropertyResolver:InformationEvent] (Info, Normal): Configuration property [StreamWriteCost] has been initialized. Source provider: LocalFileSettingsProvider. Value: 0.00024. Default: 0.00024 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [LimitNumberOfScaOpConst:InformationEvent] (Info, Normal): The number of literals in the query: 0. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [AppConfigPropertyResolver:InformationEvent] (Info, Normal): Configuration property [DwServiceSoftLimitSchemaEnabled] has been initialized. Source provider: Microsoft.SqlServer.DataWarehouse.Configuration.FeatureSwitchSettingsProvider. Value: False. Default: False [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [AppConfigPropertyResolver:InformationEvent] (Info, Normal): Configuration property [DwServiceSoftLimitQueryOptEnabled] has been initialized. Source provider: Microsoft.SqlServer.DataWarehouse.Configuration.FeatureSwitchSettingsProvider. Value: False. Default: False [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:14 AM [Thread:3652] [OptimizedStatement:InformationEvent] (Info, Normal): Distributed QO time: 1541.9893 ms [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [ExternalFile:InformationEvent] (Info, Normal): For file format DelimitedText, max # of external readers per node has been set to 8 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSPlanBuilder:InformationEvent] (Info, Normal): Table has clustered indexes, or this is a streaming plan. Parallel writers will be disabled during this operation. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [ExternalFile:InformationEvent] (Info, Normal): For file format DelimitedText, max # of external readers per node has been set to 8 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [HadoopFile:InformationEvent] (Info, Normal): Total # of files: 4, Total file splits: 4 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [ExternalFile:InformationEvent] (Info, Normal): For file format DelimitedText, max # of external readers per node has been set to 8 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSPlanBuilder:InformationEvent] (Info, Normal): [BuildExternalMoveReaders] Total 4 split lists for 1 compute nodes each can use up-to 8 external readers. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSPlanBuilder:InformationEvent] (Info, Normal): [BuildExternalMoveReaders]
Node [SQLEXT2016M01:1433]; # of readers used: 4; Reader 0, # of splits: 1; Reader 1, # of splits: 1; Reader 2, # of splits: 1; Reader 3, # of splits: 1;
 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSPlanBuilder:InformationEvent] (Info, Normal): Table has clustered indexes, or this is a streaming plan. Parallel writers will be disabled during this operation. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSPlanBuilder:InformationEvent] (Info, Normal): Table has clustered indexes, or this is a streaming plan. Parallel writers will be disabled during this operation. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSPlanBuilder:InformationEvent] (Info, Normal): Table has clustered indexes, or this is a streaming plan. Parallel writers will be disabled during this operation. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSPlanBuilder:InformationEvent] (Info, Normal): Table has clustered indexes, or this is a streaming plan. Parallel writers will be disabled during this operation. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSPlanBuilder:InformationEvent] (Info, Normal): Table has clustered indexes, or this is a streaming plan. Parallel writers will be disabled during this operation. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSPlanBuilder:InformationEvent] (Info, Normal): Table has clustered indexes, or this is a streaming plan. Parallel writers will be disabled during this operation. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSPlanBuilder:InformationEvent] (Info, Normal): Table has clustered indexes, or this is a streaming plan. Parallel writers will be disabled during this operation. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSPlanBuilder:InformationEvent] (Info, Normal): Table has clustered indexes, or this is a streaming plan. Parallel writers will be disabled during this operation. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [AppConfigPropertyResolver:InformationEvent] (Info, Normal): Configuration property [DwDmsHighResourcePlanPercentage] has been initialized. Source provider: Microsoft.SqlServer.DataWarehouse.Configuration.XdbSettingsProvider. Value: -1. Default: -1 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DmsConcurrency:InformationEvent] (Info, Normal): DwDmsHighResourcePlanPercentage is loaded: [-1]. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DmsConcurrency:InformationEvent] (Info, Normal): Check if this instance is DwService Gen1 : [False]. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSManager:InformationEvent] (Info, Normal): ExecuteCommand Query:SID335 Plan:a55cf35f-387a-46de-bb2a-4bf6791c925b [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:16 AM [Thread:3652] [DMSManager:InformationEvent] (Info, Normal): DMS Manager starting query: SID335, plan: a55cf35f-387a-46de-bb2a-4bf6791c925b. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:23 AM [Thread:4328] [DMSManager:InformationEvent] (Info, Normal): ExecuteCommand Query:SID335 Plan:61ddf166-0d34-4537-b799-09b2e7f65aa8 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:23 AM [Thread:4328] [DMSManager:InformationEvent] (Info, Normal): DMS Manager starting query: SID335, plan: 61ddf166-0d34-4537-b799-09b2e7f65aa8. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:23 AM [Thread:4296] [DMSManager:InformationEvent] (Info, Normal): Handle de-registered PBQTable{0F05B8EF-B910-4F5B-976E-40AF27BF85DA} 
8/23/2019 12:28:27 AM [Thread:4296] [CommandWorker:InformationEvent] (Info, Normal): QueryId QID510 PlanId a55cf35f-387a-46de-bb2a-4bf6791c925b StepId 3: Canceling plan due to reject threshold reached. 
8/23/2019 12:28:27 AM [Thread:3812] [CommandWorker:InformationEvent] (Info, Normal): Cancel triggered for data movement 
8/23/2019 12:28:28 AM [Thread:360] [CommandWorker:ErrorEvent] (Error, High): Plan failed with DMS CommandWorker Exception: 
Microsoft.SqlServer.DataWarehouse.Common.ErrorHandling.MppSqlException[107090:1]: 107090;Query aborted-- the maximum reject threshold (1 rows) was reached while reading from an external source: 2 rows rejected out of total 2 rows processed. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:28 AM [Thread:3652] [AbstractDataMovementExecutable`1:ErrorEvent] (Error, High): Data movement error detected- Code: 0
Microsoft.SqlServer.DataWarehouse.Common.ErrorHandling.MppSqlException[107090:1]: 107090;Query aborted-- the maximum reject threshold (1 rows) was reached while reading from an external source: 2 rows rejected out of total 2 rows processed. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:28 AM [Thread:3652] [AbstractDataMovementExecutable`1:InformationEvent] (Info, Normal): DMS Manager finishing query: SID335, plan: a55cf35f-387a-46de-bb2a-4bf6791c925b, queryId: QID510 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:28 AM [Thread:3652] [EngineInstrumentation:QueryStepExecuteErrorEvent] (Error, Critical): 
Microsoft.SqlServer.DataWarehouse.Common.ErrorHandling.MppSqlException[107090:1]: 107090;Query aborted-- the maximum reject threshold (1 rows) was reached while reading from an external source: 2 rows rejected out of total 2 rows processed. ---> Microsoft.SqlServer.DataWarehouse.Common.ErrorHandling.MppSqlException[107090:1]: 107090;Query aborted-- the maximum reject threshold (1 rows) was reached while reading from an external source: 2 rows rejected out of total 2 rows processed.
   --- End of inner exception stack trace ---
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.AbstractDataMovementExecutable`1.HandleError()
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.AbstractDataMovementExecutable`1.OnExecute(ISessionContext Session)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.AbstractExecutable`1.Execute(ISessionContext session)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.ExecutableProcessor.<>c__DisplayClass40_0.<ExecuteExecutable>b__0(IList`1 propagationTokens)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.ExecutableProcessor.ExecuteActionInDistributedTransactions(Int32 transactionsCount, Action`1 action, IList`1 propagationTokens)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.ExecutableProcessor.ExecuteExecutable(EngineExecutionPlan plan, IExecutable executable, ExecutionContext context)
   at Microsoft.Practices.TransientFaultHandling.RetryPolicy.<>c__DisplayClass1.<ExecuteAction>b__0()
   at Microsoft.Practices.TransientFaultHandling.RetryPolicy.ExecuteAction[TResult](Func`1 func)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.ExecutableProcessor.ExecutePlan(EngineExecutionPlan plan, ExecutionContext context)
   at Microsoft.Practices.TransientFaultHandling.RetryPolicy.<>c__DisplayClass1.<ExecuteAction>b__0()
   at Microsoft.Practices.TransientFaultHandling.RetryPolicy.ExecuteAction[TResult](Func`1 func)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.ExecutableProcessor.RunExecutables(EngineExecutionPlan plan, IList`1 executables)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.MultiStreamingExecutable.OnExecute(ISessionContext session) [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:28 AM [Thread:4328] [AbstractDataMovementExecutable`1:InformationEvent] (Info, Normal): DMS Manager finishing query: SID335, plan: 61ddf166-0d34-4537-b799-09b2e7f65aa8, queryId: QID510 [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:28 AM [Thread:4328] [MultiStreamingExecutable:InformationEvent] (Info, Normal): Early terminate other operations in stream. [Session.SessionId:SID335][Session.IsTransactional:False][Query.QueryId:QID510]
8/23/2019 12:28:28 AM [Thread:3812] [CommandWorker:WarningEvent] (Warning, Normal): Cannot cancel this Plan:61ddf166-0d34-4537-b799-09b2e7f65aa8, due to the uncancellable State:Completed 
8/23/2019 12:28:28 AM [Thread:3812] [CommandWorker:InformationEvent] (Info, Normal): Cancel triggered for data movement 
8/23/2019 12:28:28 AM [Thread:3652] [MultiStreamingExecutable:InformationEvent] (Info, Normal): One or more exceptions hit. Generating aggregated exception summary
===========================================
Exception summary
Exception Message: 107090;Query aborted-- the maximum reject threshold (1 rows) was reached while reading from an external source: 2 rows rejected out of total 2 rows processed.
Exception stack trace
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.AbstractDataMovementExecutable`1.HandleError()
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.AbstractDataMovementExecutable`1.OnExecute(ISessionContext Session)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.AbstractExecutable`1.Execute(ISessionContext session)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.ExecutableProcessor.<>c__DisplayClass40_0.<ExecuteExecutable>b__0(IList`1 propagationTokens)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.ExecutableProcessor.ExecuteActionInDistributedTransactions(Int32 transactionsCount, Action`1 action, IList`1 propagationTokens)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.ExecutableProcessor.ExecuteExecutable(EngineExecutionPlan plan, IExecutable executable, ExecutionContext context)
   at Microsoft.Practices.TransientFaultHandling.RetryPolicy.<>c__DisplayClass1.<ExecuteAction>b__0()
   at Microsoft.Practices.TransientFaultHandling.RetryPolicy.ExecuteAction[TResult](Func`1 func)
   at Microsoft.SqlServer.DataWarehouse.Engine.Executables.ExecutableProcessor.ExecutePlan(EngineExecutionPlan plan, ExecutionContext context)
   at Microsoft.Practices.TransientFaultHandling.RetryPolicy.<>c__DisplayClass1.<ExecuteAction>b__0()

I would like know the reason for the error, which is not very clear from any of the error log files.

↧

SQL Database Error 924 – Database is already open and can only have one user at a time

August 4, 2019, 11:22 pm

≫ Next: Stopat restore

≪ Previous: Polybase SQL Server 2019 : External Table Query Result Issue

In a Java-application I am working on, I have an updater, which updates the db structure whenever a new version is started for the first time.
This updater is executed in single user mode, to ensure no one is working on the db in the meantime.
During this update, I have to request meta data and this request sometimes leads to the Error 924.
This bug occurs only since I upgraded the mssql-jdbc driver to 6.3.3 or later and I therefor opened an issue on theirGithub page.
However, during the discussion in this issue, it turned out, that neither the driver nor our code opens a new connection. Even the SQL Server Profiler logs only one connection.
During my research, I found an article, which states, that SINGLE_USER can only be used if AUTO_UPDATE_ASYNC is turned OFF. That's the case in my database, but since async jobs on the SQL Server seem to cause problems with single user mode, my guess is, that some statements used by the driver to load the meta data (starting with Line 988) are executed asynchronously. This presumption is reinforced by the fact, that the error only occurs, if the database is stored on a HDD drive. On SSD drives everything works fine, which sounds like some kind of race condition.
Our current workaround for this is to catch this error and retry the statement after 3 seconds, until it works. Usually it takes about 5 retries or 15 seconds.

Is there some setting in the SQL Server I am missing or what else could cause this issue?

↧

Stopat restore

August 26, 2019, 2:00 am

≫ Next: determining impact of a new index on 6 views

≪ Previous: SQL Database Error 924 – Database is already open and can only have one user at a time

To restore the database to a point of time I have been using full backup, differential backup and log backup. Storing all these backups needs space. I was thinking stopat restore could help because if we could restore to a point of time from full backup other backups would not be needed but stopat restore works only with log backup. In a sequence of full, differential and log backup each log backup contains changes after the last backup. Is it possible to take log backup since last full backup by skipping the differential and log backups?

↧

determining impact of a new index on 6 views

August 26, 2019, 7:48 am

≫ Next: GetDate() in SQL Server on Linux and UTC

≪ Previous: Stopat restore

Hi we run 2017 STD. Long story but I want to see the impact of a new index on one table (3.3 million rows) that is referenced along with 12 other tables in 6 views.

One of the other tables is gigantic, so i'm reluctant to mimic (it may be impossible) the situation in development. But I would probably get away with copying the table that's getting the new index to a temp table in prod, building indexes over it and running the views as queries with that table substituted.

I don't know if running stats on a temp table is a valid thing but I'd probably want to do that on the temp table and its indexes if possible.

I might then run exec plans, queries etc to judge if the new index is going to impact the 6 views.

Is this a valid approach / alternative to measuring impact of a new index ? Is there a better way in sql server to project the impact of a new index? what if I might want to replace an existing index on the table with the new index?

I can pretty easily measure the impact on writes to the table in question so that isn't an issue.

↧

GetDate() in SQL Server on Linux and UTC

July 10, 2019, 4:02 am

≫ Next: Extended event capture

≪ Previous: determining impact of a new index on 6 views

Hi,

Is my understanding correct that in SQL Server on Linux we will always get UTC time when running SELECT GetDate()?

Can't find any documentation related to this at all.

Thanks

↧

Extended event capture

August 27, 2019, 1:54 am

≫ Next: SQL Server database Parameter List

≪ Previous: GetDate() in SQL Server on Linux and UTC

Hi All,

I want to capture deadlock information and associated batch and tsql stmts . I tried setting up extended event session.
I was able to reproduce the deadlock error but not sure why xml deadlock graph was not getting captured in the .xel file.
Am I doing anything wrong here? Is track_causality option = ON is doing something wrong? Please suggest.

CREATE EVENT SESSION [xe_deadlocks] ON SERVER
ADD EVENT sqlserver.error_reported(
    ACTION(sqlserver.client_app_name,sqlserver.client_hostname,sqlserver.database_id,sqlserver.database_name,sqlserver.plan_handle,sqlserver.query_plan_hash,sqlserver.server_principal_name,sqlserver.session_id,sqlserver.sql_text,sqlserver.transaction_id)
    WHERE ([sqlserver].[database_name]=N'testdb' AND [sqlserver].[is_system]=(0))),
ADD EVENT sqlserver.rpc_completed(
    ACTION(sqlserver.client_app_name,sqlserver.client_hostname,sqlserver.database_id,sqlserver.database_name,sqlserver.plan_handle,sqlserver.query_plan_hash,sqlserver.server_principal_name,sqlserver.session_id,sqlserver.sql_text,sqlserver.transaction_id)
    WHERE ([sqlserver].[database_name]=N'testdb' AND [sqlserver].[is_system]=(0))),
ADD EVENT sqlserver.sp_statement_completed(
    ACTION(sqlserver.client_app_name,sqlserver.client_hostname,sqlserver.database_id,sqlserver.database_name,sqlserver.plan_handle,sqlserver.query_plan_hash,sqlserver.server_principal_name,sqlserver.session_id,sqlserver.sql_text,sqlserver.transaction_id)
    WHERE ([sqlserver].[database_name]=N'testdb' AND [sqlserver].[is_system]=(0))),
ADD EVENT sqlserver.sql_batch_completed(
    ACTION(sqlserver.client_app_name,sqlserver.client_hostname,sqlserver.database_id,sqlserver.database_name,sqlserver.plan_handle,sqlserver.query_plan_hash,sqlserver.server_principal_name,sqlserver.session_id,sqlserver.sql_text,sqlserver.transaction_id)
    WHERE ([sqlserver].[database_name]=N'testdb' AND [sqlserver].[is_system]=(0))),
ADD EVENT sqlserver.xml_deadlock_report(
    WHERE ([sqlserver].[database_name]=N'testdb' AND [sqlserver].[is_system]=(0)))
ADD TARGET package0.event_file(SET filename=N'z:\xevents\xe_deadlocks.xel',max_file_size=(1024),max_rollover_files=(4))
WITH (MAX_MEMORY=4096 KB,
EVENT_RETENTION_MODE=ALLOW_SINGLE_EVENT_LOSS,
MAX_DISPATCH_LATENCY=30 SECONDS,
MAX_EVENT_SIZE=0 KB,
MEMORY_PARTITION_MODE=NONE,
TRACK_CAUSALITY=ON
,STARTUP_STATE=OFF
)
GO

I was trying to pull deadlock information using below sql stmts.But I dont see deadlock information is getting logged.

drop table #results
CREATE TABLE #results
(RowID INT IDENTITY PRIMARY KEY,
event_data XML);

go

DECLARE @Filename VARCHAR(255) = N'z:\xevents\xe_deadlocks*.xel';
-- note: get the path from above serverproperty('errorlogfilename') . This is where system_health session log files are resided

INSERT INTO #results (event_data)
-- Cast the target_data to XML
SELECT CAST(event_data AS XML) AS event_data
FROM sys.fn_xe_file_target_read_file(@Filename, null, null, null)

go

select * from #results

/* Output Deadlock Graphs */
SELECT
event_data.value('(event/@timestamp)[1]', 'DATETIME2') AS eventtimestamp,
event_data.query('(event/data/value/deadlock)[1]') AS DeadlockGraph
FROM #results
WHERE event_data.exist('event[@name="xml_deadlock_report"]') = 1
ORDER BY eventtimestamp ASC
OPTION(MAXDOP 1);

Thanks,
Sam

↧

SQL Server database Parameter List

August 27, 2019, 2:21 am

≫ Next: Issue shrinking "empty" file using DBCC SHRINKFILE

≪ Previous: Extended event capture

Hi Sir,

Is there any way to get ALLdatabase parameter list by using DATABASEPROPERTYEX or any other sql.

DATABASEPROPERTYEX('AdventureWorks2012','Version') AS [Version]

Regards,

Deepak

↧

Issue shrinking "empty" file using DBCC SHRINKFILE

August 27, 2019, 2:28 pm

≫ Next: does selecting with nolock from a replicated db increase significantly the chances of a duplicate being selected?

≪ Previous: SQL Server database Parameter List

Hello all.

We could use some help with an issue that we haven't seen before.

When attempting to shrink a data file where we have migrated the tables off of the associated filegroup, we are getting the following error:

DBCC SHRINKFILE: Page 5:458510930 could not be moved because it belongs to an active online index build with LOBs.

Using DBCC PAGE to determine what this page belongs to, we find that it belongs to an object_id that doesn't actually exist in sys.objects.

So there are allocation units for a table that doesn't actually exist.

We have run DBCC CHECKDB on the database which doesn't report any corruption and we have actually backed up and restored the database to another server hoping that would clear up any "orphaned" pages, etc., but nothing has gotten us past this problem.

Any insight would helpful.

↧

does selecting with nolock from a replicated db increase significantly the chances of a duplicate being selected?

August 27, 2019, 7:51 am

≫ Next: DBCC loginfo

≪ Previous: Issue shrinking "empty" file using DBCC SHRINKFILE

Hi we run std 2017. It only happens 2 or 3 times a month but even that seems to be more frequent than any of the anomalies I've read about that nolock brings to the "table".

We have a select (maybe 1000-2000 records) on a replicated table with nolock based on an entry date being greater than the last high entry date taken in the previous hour. I'm sure it runs sometimes while replication is occurring. The serially assigned PK of the replicated table guarantees dupes aren't physically stored on the table. We have some pretty convincing evidence that duplicates are being selected now and then. The PK isn't clustered and on the surface the index that is clustered while unique doesn't appear to be correlated to "time". I'm looking closer at that.

Does the tandem of replication and reading with nolock increase the possibility of dupes so significantly that we might see them more often than some of the other anomalies nolock creates?

We select on the hour from 20 similar databases, about 6 at a time, all of which sit on the same server, all of which are replication targets and at first glance all seem to be on the same physical device . As usual I'm concerned about locking if we take the nolock off.

↧

DBCC loginfo

August 27, 2019, 7:18 pm

≫ Next: How to track the difference data column while comparing with 2 tables having same column names

≪ Previous: does selecting with nolock from a replicated db increase significantly the chances of a duplicate being selected?

I am curious on the following situation.

1. I have a database in FULL recovery mode

2 I performed database log backup.

3. there are no more active transaction or replication

4. Check in DBCC loginfo for that database, the status of the VLF are still 2 . why is that ?

↧

How to track the difference data column while comparing with 2 tables having same column names

August 28, 2019, 2:17 am

≫ Next: Open transaction issue on DB and proactive measures and suggestions needed

≪ Previous: DBCC loginfo

I have 2 tables Snapshot table and temptable. temptable is holding copy of snapshot table before snapshot gets incremental load. My objective is to pull the Rowid & column and value which ever has difference compare to each other.

So I need Some code or the ways or articles to implement this.

Snapshot Table:

RowIDObject Column2 Column31            Jack   USA2            Ryan   GER

Temp Table:

RowIDObject Column2 Column31            Jack   USA2            Ryan   GER

Snapshot Table After Incremental load

RowIDObject Column2 Column31            Jack   UAE2            Ryan1   GER

I want to hold in a new table

RowID ColumnChanged ActualValue Updated Value1       Column3       USA         UAE2       Column2       Ryan        Ryan1

Note: I am using Azure SQL DB.

↧

Open transaction issue on DB and proactive measures and suggestions needed

August 28, 2019, 3:24 am

≫ Next: do views automatically pick up rerun stats etc on tables and indexes

≪ Previous: How to track the difference data column while comparing with 2 tables having same column names

I understood that

Seclects are not loggged in Transaction Logs; only DML (INSERT, UPDATE, DELETE, Merge) or Any other DDL (CREATE, DROP, ALTER etc.) statements are logged.

howerver yesterday during an issue with my application user we did saw an open transaction on the DB for quite an long time ( 20 minutes), this query is an Select statement with an left outer join and my application user argues that it was an common query that is called umpteen times and was all along fine except that on that particular eventful time it was creating an issue for them they rebooted the workstation that was runnign the query but still the session was not remvoed from DB

question 1 to us is why did not the SPID got removed in the DB when the workstation that was issuing the SELECT was rebooted as we had it forefully KILLED on DB layer, where is the issue on application layer or in DB engine ??

Question 2 to us is if there is an alerting mechanism or an event ID for open transactions to be monitored and alerted via SCOM tool if so what is the Enterprise monitorinig threshold for that parameter ( application team is arguing that no open transactions should exist and should get completed in milliseconds )

Is there any other email alert mechanism or proactive measure to alert us when the open transaction on the Db is for more time.

Appreciate all your help on this requirement to gracefully address the concerns

Regards and Wishes

Eben

Query -

SELECT [t0].[row_version] AS [RowVersion], [t0].[organization_guid] AS [__Key], [t0].[name] AS [Name], [t0].[comments] AS [Comments], [t1].[row_version] AS [RowVersion2], [t1].[organization_guid] AS [__OrganizationKey], [t1].[site_guid] AS [__Key2], [t1].[name] AS [Name2], [t1].[person_id_type_guid] AS [__PersonIdTypeKey], [t1].[default_inbox_guid] AS [__DefaultInboxKey], [t1].[comments] AS [Comments2], [t1].[domain_name] AS [DomainName], [t1].[default_template_guid] AS [__DefaultTemplateKey], [t1].[shortcuts] AS [Shortcuts], (
SELECT COUNT(*)
FROM [sites_view] AS [t2]
WHERE [t0].[organization_guid] = [t2].[organization_guid]
) AS [value]
FROM [organizations_view] AS [t0]
LEFT OUTER JOIN [sites_view] AS [t1] ON [t0].[organization_guid] = [t1].[organization_guid]
ORDER BY [t0].[organization_guid], [t1].[site_guid

↧

do views automatically pick up rerun stats etc on tables and indexes

August 28, 2019, 5:07 am

≫ Next: Extended Events and Perfmon analyze together

≪ Previous: Open transaction issue on DB and proactive measures and suggestions needed

Hi We run 2017 std. Just to be safe I am wondering if following an index add on 2 tables and removal of an index from 1 table if the views using those tables require some sort of recompile? Before the index adds, an important search date was converted from nulls to 12/31/9999 also. I ran stats on the tables and indexes. Just wondering if views will pick up on the changes seamlessly. I know estim exec plans saw the changes but that was for adhoc queries, not views.

↧

Extended Events and Perfmon analyze together

August 28, 2019, 9:07 pm

≫ Next: XQuery Cluster Diagnostic files (SQLDiag)

≪ Previous: do views automatically pick up rerun stats etc on tables and indexes

In SQL 2014, I need to analyze XE files containing stored procedures/queries generating high load and see the impact on Perfmon data.

I am interested in CodePlex and/or MS tools to analyze together XE and Perfmon, but not a third-partysubscription.

AFAIK there was nothing available a few years back, if no tool available then any blog/hyperlink for this?

Thank you,

↧

XQuery Cluster Diagnostic files (SQLDiag)

August 22, 2019, 12:16 pm

≫ Next: SQL Server 2014 not connecting when TLS 1.0 is disabled

≪ Previous: Extended Events and Perfmon analyze together

I need to parse Cluster Diagnostic files.

I need the query to bring all the distinct names from data elements of one of event type (lets say "scheduler_monitor_non_yielding_ring_buffer_recorded" ) under the RingBufferTarget . ex:

Event Name Attribute

================================== =============
scheduler_monitor_non_yielding_ring_buffer_recordedid
scheduler_monitor_non_yielding_ring_buffer_recorded timestamp
scheduler_monitor_non_yielding_ring_buffer_recorded opcode
......

Shortened XML below:

<event name="component_health_result" package="sql_cluster_resource" timestamp="2019-08-12T14:06:12.454Z">
<data name="state">
<value>0</value>
</data>
<data name="failure_condition_level">
<value>3</value>
</data>
<data name="creation_time">
<value>2019-08-12 10:06:12.453</value>
</data>
<data name="component_type">
<value>instance</value>
</data>
<data name="component">
<value>events</value>
</data>
<data name="state_desc">
<value>unknown</value>
</data>
<data name="data">
<value>
<events>
<session startTime="2019-07-13T01:55:50.300" droppedEvents="0" largestDroppedEvent="0">
<RingBufferTarget truncated="0" processingTime="3891" totalEventsProcessed="46481" eventCount="43" droppedCount="0" memoryUsed="24560">
<event name="scheduler_monitor_non_yielding_ring_buffer_recorded" package="sqlos" timestamp="2019-07-31T13:42:46.283Z">
<data name="id">
<type name="uint32" package="package0" />
<value>0</value>
</data>
<data name="timestamp">
<type name="uint64" package="package0" />
<value>1604342839</value>
</data>
<data name="opcode">
<type name="event_opcode" package="sqlos" />
<value>0</value>
<text>Begin</text>
</data>
<data name="node_id">
<type name="uint16" package="package0" />
<value>0</value>
</data>
<data name="scheduler">
<type name="uint32" package="package0" />
<value>5</value>
</data>
<data name="worker">
<type name="ptr" package="package0" />
<value>0x00000005238f8160</value>
</data>
<data name="yields">
<type name="uint64" package="package0" />
<value>195936512</value>
</data>
<data name="worker_utilization">
<type name="uint32" package="package0" />
<value>99</value>
</data>
<data name="process_utilization">
<type name="uint32" package="package0" />
<value>25</value>
</data>
<data name="system_idle">
<type name="uint32" package="package0" />
<value>66</value>
</data>
<data name="user_mode_time">
<type name="uint64" package="package0" />
<value>100156250</value>
</data>
<data name="kernel_mode_time">
<type name="uint64" package="package0" />
<value>0</value>
</data>
<data name="page_faults">
<type name="uint32" package="package0" />
<value>12</value>
</data>
<data name="working_set_delta">
<type name="int64" package="package0" />
<value>8192</value>
</data>
<data name="memory_utilization">
<type name="uint32" package="package0" />
<value>99</value>
</data>
<data name="call_stack">
<type name="callstack" package="package0" />
<value />
</data>
</event>
<event name="scheduler_monitor_non_yielding_ring_buffer_recorded" package="sqlos" timestamp="2019-07-31T13:42:51.298Z">
<data name="id">
<type name="uint32" package="package0" />
<value>0</value>
</data>
<data name="timestamp">
<type name="uint64" package="package0" />
<value>47382888</value>
</data>
<data name="opcode">
<type name="event_opcode" package="sqlos" />
<value>1</value>
<text>End</text>
</data>
<data name="node_id">
<type name="uint16" package="package0" />
<value>0</value>
</data>
<data name="scheduler">
<type name="uint32" package="package0" />
<value>5</value>
</data>
<data name="worker">
<type name="ptr" package="package0" />
<value>0x00000005238f8160</value>
</data>
<data name="yields">
<type name="uint64" package="package0" />
<value>195936526</value>
</data>
<data name="worker_utilization">
<type name="uint32" package="package0" />
<value>99</value>
</data>
<data name="process_utilization">
<type name="uint32" package="package0" />
<value>25</value>
</data>
<data name="system_idle">
<type name="uint32" package="package0" />
<value>66</value>
</data>
<data name="user_mode_time">
<type name="uint64" package="package0" />
<value>100156250</value>
</data>
<data name="kernel_mode_time">
<type name="uint64" package="package0" />
<value>0</value>
</data>
<data name="page_faults">
<type name="uint32" package="package0" />
<value>12</value>
</data>
<data name="working_set_delta">
<type name="int64" package="package0" />
<value>8192</value>
</data>
<data name="memory_utilization">
<type name="uint32" package="package0" />
<value>99</value>
</data>
<data name="call_stack">
<type name="callstack" package="package0" />
<value />
</data>
</event>
<event name="security_error_ring_buffer_recorded" package="sqlserver" timestamp="2019-08-08T19:47:51.170Z">
<data name="id">
<type name="uint32" package="package0" />
<value>0</value>
</data>
<data name="timestamp">
<type name="uint64" package="package0" />
<value>47264855</value>
</data>
<data name="session_id">
<type name="int16" package="package0" />
<value>82</value>
</data>
<data name="error_code">
<type name="uint32" package="package0" />
<value>1332</value>
</data>
<data name="api_name">
<type name="unicode_string" package="package0" />
<value>LookupAccountSid</value>
</data>
<data name="calling_api_name">
<type name="unicode_string" package="package0" />
<value>LookupAccountSidInternal</value>
</data>
<data name="call_stack">
<type name="callstack" package="package0" />
<value />
</data>
</event>
<event name="security_error_ring_buffer_recorded" package="sqlserver" timestamp="2019-08-08T19:48:41.554Z">
<data name="id">
<type name="uint32" package="package0" />
<value>0</value>
</data>
<data name="timestamp">
<type name="uint64" package="package0" />
<value>47264855</value>
</data>
<data name="session_id">
<type name="int16" package="package0" />
<value>82</value>
</data>
<data name="error_code">
<type name="uint32" package="package0" />
<value>1332</value>
</data>
<data name="api_name">
<type name="unicode_string" package="package0" />
<value>LookupAccountSid</value>
</data>
<data name="calling_api_name">
<type name="unicode_string" package="package0" />
<value>LookupAccountSidInternal</value>
</data>
<data name="call_stack">
<type name="callstack" package="package0" />
<value />
</data>
</event>
</RingBufferTarget>
</session>
</events>
</value>
</data>
<data name="instance_name">
<value>InstanceName</value>
</data>
<data name="node_name">
<value>ServerName</value>
</data>
</event>

↧

SQL Server 2014 not connecting when TLS 1.0 is disabled

August 28, 2019, 7:47 am

≫ Next: How to get information "Who has executed SQL Server Profiler"

≪ Previous: XQuery Cluster Diagnostic files (SQLDiag)

Hi Team,

We are unable to connect to SQL server 2014 (https://www.microsoft.com/en-us/download/details.aspx?id=51187) when TLS 1.0 is disabled in registry. Connection is successful only when TLS 1.2 is enabled in registry and TLS 1.0 registry key is not present.

Below is error which we are getting

com_error: Unspecified error;[DBNETLIB][ConnectionOpen (SECDoClientHandshake()).]SSL Security error.) “

Please suggest how to enable connection with TLS 1.0 disabled

↧