As part of my data import, it has become a requirement to fire a URL for each "new" record, after the de-duplication process... ideally (but not necessarily) from within the same SSIS package. This will create the record in another system.I have never attempted to do anything like this, and am not sure exactly where to start.Each URL is unique for each "new" record. I am using a SQL Task to generate the URLs for each new record, and store them in a table.So what I need is some method of invoking the URLs in a database table. Does anyone have a recommendation about how I can make a start on this?Thanks in advance.
↧
Invoke URL from SSIS?
↧
can't able to get the data from the remote server
Hi,I can't able to get the data from remote server.I ma getting only headerrs not getting data.it 'll throwing below error:System.Net.WebException: The remote server returned an error: (500) Internal Ser ver Error. at System.Net.WebClient.File(Uri address, String fileName) at System.Net.WebClient.File(String address, String fileName) at Script.Program.Main(String[] args) Thanks.
↧
↧
Sharepoint List Source Error -A possible reason might be you are trying to retrieve too many items at a time (Batch size)
[b][/b]I am trying to get data from a Sharepoint List using the following CAML Query in SharePoint List Source<Query><Where><Contains><FieldRef Name='Function Affected'/><Value Type='Text'>IT</Value></Contains></Where></Query>I am able to query some other columns from the same SharePoint List without any error msgI have set the Batch Size to values between 10 and 1000, but still am getting the following error message.[size="1"][SharePoint List Source [1]] Error: Microsoft.Samples.SqlServer.SSIS.SharePointUtility.SharePointUnhandledException: Unspecified SharePoint Error. A possible reason might be you are trying to retrieve too many items at a time (Batch size) ---> System.ServiceModel.FaultException: Exception of type 'Microsoft.SharePoint.SoapServer.SoapServerException' was thrown. Server stack trace: at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc) at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout) at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs) at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation) at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message) Exception rethrown at [0]: at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg) at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type) at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.ListsService.ListsSoap.GetListItems(GetListItemsRequest request) at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.ListsService.ListsSoapClient.ListsService_ListsSoap_GetListItems(GetListItemsRequest request) at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.ListsService.ListsSoapClient.GetListItems(String listName, String viewName, XElement query, XElement viewFields, String rowLimit, XElement queryOptions, String webID) at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.Adapter.ListsAdapter.GetSharePointListItems(String listName, String viewId, XElement queryXml, XElement viewXml, Int16 pagingSize, XElement queryOptionsXml) --- End of inner exception stack trace --- at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.Adapter.ListsAdapter.GetSharePointListItems(String listName, String viewId, XElement queryXml, XElement viewXml, Int16 pagingSize, XElement queryOptionsXml) at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.Adapter.ListsAdapter.GetSharePointListItemData(String listName, String viewId, IEnumerable`1 fieldNames, XElement query, Boolean isRecursive, Int16 pagingSize) at Microsoft.Samples.SqlServer.SSIS.SharePointUtility.ListServiceUtility.GetListItemData(Uri sharepointUri, NetworkCredential credentials, String listName, String viewName, IEnumerable`1 fieldNames, XElement query, Boolean isRecursive, Int16 pagingSize) at Microsoft.Samples.SqlServer.SSIS.SharePointListAdapters.SharePointListSource.PrimeOutput(Int32 outputs, Int32[] outputIDs, PipelineBuffer[] buffers) at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostPrimeOutput(IDTSManagedComponentWrapper90 wrapper, Int32 outputs, Int32[] outputIDs, IDTSBuffer90[] buffers, IntPtr ppBufferWirePacket)[/size]
↧
Login timeout when trying to add an existing package on SQL Server to a solution in VS
I have an integration server where I can connect to in SSMS without any problem. But when I try to add an existing package on the same integration server to a solution in VS 2008 I get a login time out. Error follows. Does anyone know what the issue may be?TITLE: Add Copy of Existing Package------------------------------An OLE DB error 0x80004005 (Login timeout expired) occurred while enumerating packages. A SQL statement was issued and failed.------------------------------ADDITIONAL INFORMATION:An OLE DB error 0x80004005 (Login timeout expired) occurred while enumerating packages. A SQL statement was issued and failed.------------------------------BUTTONS:OK------------------------------
↧
Remtoe SQL Server backup on local server using SSIS
I am trying to create a package that would take remote sql server backup on my local server. But all efforts went in no-result. Does anyone has any idea to create the same package ?Thanks in advance...
↧
↧
Stopping an SSIS Package on Data Flow Fail
I have an SSIS package with 5 separate data flow tasks. They are not linked and run in parallel. I;m trying to figure out how to stop the other data flow tasks when one task fails. Anyone got any idea? I tried setting Max Fails etc but doesn't seem to stop the other tasks.
↧
is it possible to save data in excel by user id
is it possible to save data in excel by userID and also need to email the file tooHere is the sample data User ID | Email | Product Info | Price---------------------------------------23456 | a@hotmail.com | ABC | 23.0023456 | a@hotmail.com | ABC | 23.0023456 | a@hotmail.com | ABC | 23.0012345 | d@hotmail.com | ABC | 23.0012345 | d@hotmail.com | ABC | 23.00 ....
↧
Problem parsing expression
I'm trying to create an expression for a flat file connector connection string. I'm in SSIS 2008. here's the code:"G:\Mhub_Operations\DATA\SSIS\CiscoFAST\" +@[User::FileName_Prefix] +@[User::FileName_Region] +@[User::FileName_Timestamp] +".txt"and here's the error message.Attempt to parse the expression ""G:\Operations\DATA\SSIS\AST\" &@[User::FileName_Prefix] &@[User::FileName_Region] & @[User::FileName_Timestamp] & ".txt""failed. The token “.” At line number 5, character 2 was not recognized. The expression cannot be parsed because it contains invalid elements at the location specified.When I take out that ".txt", I get:Attempt to parse the expression ""G:\Operations\DATA\SSIS\AST\" +@[User::FileName_Prefix] +@[User::FileName_Region] +@[User::FileName_Timestamp] " failed. The token """ at line number "0", character number "0" was not recognized. The expression cannot be parsed because it contains invalid elements at the location specified.Any ideas how to work this?
↧
how to load oracle database data to sql server with the help of SSIS
how to load oracle database data to sql server with the help of SSIS
↧
↧
Which would be faster, an SSIS Foreach or SQL Cursor for 6300 rows?
I'm currently re-writing an inherited C#/SQL CSV import process due to speed issues with the previous process (reading each line with a C# StreamReader).The CSV has 6300 rows, and 21 columns (mostly strings, a couple of dates), which are used to update figures in our internal database.I'm currently unsure of which of the below ideas would be more efficient:- A Foreach loop on all 21 columns, passing the 21 parameters to an Execute SQL Task and using to a Stored Procedure to update each row- Modifying my Data Flow Control to dump the imported result set to an SQL table, and using an SQL Cursor to iterate through each rowI expect a cursor would be much more efficient than starting 6300 SSIS seperate Execute SQL tasks and passing all 21 parameters to it for each iteration of the foreach loop, but am unsure of this. Does anyone have any insight or guidance over which approach is more efficient? Any help would be greatly appreciated.
↧
Problem writing to user variable
I can't seem to get this to write to a user variable whose scope is at the package level. If I set breakpoints, the variables (local and user) are being set to an expected value. It just won't show in the variables list.Imports SystemImports System.DataImports System.MathImports Microsoft.SqlServer.Dts.Runtime<System.AddIn.AddIn("ScriptMain", Version:="1.0", Publisher:="", Description:="")> _<System.CLSCompliantAttribute(False)> _Partial Public Class ScriptMain Inherits Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase Enum ScriptResults Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure End Enum Public Sub Main() Dim Timestamp As String ' ' Add your code here ' Timestamp = Now.Year.ToString() & Now.Month.ToString() & Now.Day.ToString() & Now.Minute.ToString() & Now.Second.ToString + Now.Millisecond.ToString Dts.Variables.Item("User::FileName_Timestamp").Value = Timestamp Dts.TaskResult = ScriptResults.Success End SubEnd Class
↧
Config files priority
I have a package that checked the enable configurations box to enable config files but I don't have a config listed in the package. The ssis pkg is deployed to the server and is called from a SQL Agent Job. The Job has the config set in the configurations tab. I renamed the actual file that the job will try to set the configuration to to try to get an error to ensure Im usiong the correct file. I'm getting an error stating that select permission is denied. Shouldn't I be getting an error saying that the config file couldn't be found?
↧
SSIS 2012 - How to Query Excel File Using "Execute SQL Task"
First of all, I don't see a SSIS 2012 forum here, just this SSIS 2005 forum, so I hope it's o.k. that I'm posting in this particular forum since it's the closest to what I'm actually looking for...I'm developing a SSIS 2012 solution for a client. The gist of the project is that it basically runs once daily to loop through subfolders present in a particular path. If new Excel files are found to have been placed in the various subfolders since the SSIS package ran 24 hours earlier, then the process runs, loops through the various Excel files which are all different structurally, and does ETL on all of the files and places the transformed data into individual Excel spreadsheets (via the use of an Excel template file) to where data from each Excel file now follows the same Excel file structure.There more to the project than what I stated above, but that's the main component of the project.My client has requested an email to be sent along with inclusion of a few columns of data from each originating Excel file based on the following criteria. Two of the fields commonly provided in each Excel file are "Hire Date" and "Rehire Date". I'm needing to create what I assume would be an Execute SQL Task as each original Excel file is looped through, and if there's a "Rehire Date" present that's greater than the "Hire Date" in each row, the client would like to receive an email with just a few columns of information (Person's Name, Social Security#, Hire Date, & Rehire Date) to sort of serve as a flag for them to call the provider of the specific Excel files where the above criteria has found to exist to verify with them that the "Hire Date" and "Rehire Date" data was provided correctly.The situation is complicated by the fact that the Worksheet Names for each of the incoming Excel files can be different file to file. I've established some coding that dynamically determines what each Excel file's worksheet names are, so I'm hopeful I can somehow use that variable in constructing the SQL statement something like "Select * from [User::vSheetName] where Rehire Date > Hire Date". Is that possible?I'd also need to perform a count of the above query to where if there are 0 results returned, the sending of the desired email doesn't occur. The resulting couple of columns from the query in the email don't of course need to be sent until the situation of "Rehire Date > Hire Date" is actually found to produce at least one record via the query.Lastly, due again to the fact that all of the incoming Excel files are different in many cases, many of the Excel files don't even have the "Rehire Date" field provided, and if that's the case, I'd need to somehow determine that this particular field isn't even present in the Excel file and therefore this Execute SQL Task doesn't even need to be processed since the "Rehire Date" field isn't even a part of the original Excel file.I've found some decent examples online that deal with writing Execute SQL Tasks to query against an underlying SQL Server database, but I'm not finding many examples that delve into any special nuances that exist when you're trying to query from an Excel spreadsheet via an Execute SQL Task.Many thanks for any clarification / examples / links to good examples anyone can provide here.
↧
↧
Update of two tables at the same time
Hi @ ALLbefore i explain my problem, below situation at the momenti have a simple data flow task, with a OLE Source, Search and a target OLE.I´m just checking the values in the target, by not existing a new row is getting inserted.What i want to have is:by not existing i want to make additionally an update in another table (in the same DB) at the same time, toomaybe two red arrows could make it happen but i know i don`t have these chancemaybe with variables (but variables are too complicated for me)maybe with event handlers?? (i don`t know)..........Is there any chance to realize that with SQL SERVER 2008thanks for any help
↧
(SSIS 2012) - Develop on 32-bit machine, Install on 64-bit machine confusion
I'm developing a comprehensive SSIS solution for a client. Development is being done on a virtual machine (VMWare) that's a 32-bit environment.The main gist of the project is that the SSIS package will be looking for and accessing / manipulating a high volume of Excel spreadsheets nightly. Most of the Excel spreadsheets are saved using the most recent version (MS Office 2013) of Excel. On the development machine, I've installed the Microsoft Access Database Engine 2010 Redistributable (32-bit version). I'm using the "Excel Connection Manager" to allow the package to connect to the numerous Microsoft Excel 2013 spreadsheets (.xlsx). Everything's working fine. My connection string used for my "Source" and "Destination" Excel Connection Managers is basically:Provider=Microsoft.ACE.OLEDB.12.0;Data Source=C:\MyProject\ExcelFile.xlsx";Extended Properties="EXCEL 12.0 XML;HDR=YES";On the client's 64-bit SQL Server machine where the finished solution will ultimately be installed, the machine is *NOT* running Microsoft Office. It's your "standard" situation I guess as far as a machine that's intended solely to run SQL Server and doesn't have any extra stuff like Microsoft Office / etc. installed on there.The SSIS package will be called on the client's 64-bit machine via a SQL Server Agent job.I've read and read forum posts online about the 32-bit/64-bit situation with SSIS and am frankly very confused as to what (if any) changes/obstacles I need to be prepared for once I ultimately install the finished solution on my client's 64-bit SQL Server machine.Questions:(1) The "Run64BitRuntime" property for my SSIS solution in my 32-bit development environment is currently set as "False". Does this need to be changed to "True" before I build/deploy the solution for installation on my client's 64-bit SQL Server machine?(2) Again, I have the 32-bit version of the Microsoft Access Database Engine 2010 Redistributable installed on my development machine. Once I install the finished solution on my client's 64-bit SQL Server machine, do I instead need to install the *64-bit* version of the Microsoft Access Database Engine 2010 Redistributable on the client's machine? Or, do I need to install the 32-bit Microsoft Access Database Engine 2010 Redistributable on the client's SQL Server machine? This is the most confusing point to me, as everything I read states that SSIS 2012 isn't a 64-bit application. Very confusing as to which (32-bit or 64-bit) Microsoft Access Database Engine 2010 Redistributable needs to be installed on the client's 64-bit machine with the conflicting info. I've read regarding 32/64-bit issues as related to SSIS.Many thanks for reading through my detailed post/questions. I'm just trying to think ahead and make sure I'm going down the right path with my ongoing development as to allow the package that's running fine at present in my 32-bit development environment to continue to run successfully once it's installed on the client's 64-bit environment, has to still deal with Excel 2013 Excel files residing in that production 64-bit environment, etc.
↧
(SSIS 2012) Excel Destination Columns Added After Data Flow Task Already Designed
Working on an SSIS 2012 project for a client.The project basically finds and loops through a bunch of incoming Excel file, does ETL on each Excel file, and places the transformed/cleaned data into individual Excel template files that just have column headers in them until the data from each Excel file is imported into the template files.The Excel template file that's serving as the Excel Destination is generally staying the same as far as the number and order of columns expecting incoming data, but it is changing occasionally as my client finalizes what fields of data they want to be able to be captured in the template files.So, for simplicity, let's say the package and associated data flow tasks are designed at present with these 3 fields in place in the Excel template file:--- Name--- Social Security #--- Phone #If my client decides after everything's already been designed that they also want to add a 4th column ("Date of Birth") to the Excel template file, I'm seemingly having to recreate all components involved in the data flow tasks in order to be able to refresh and see/map this new "Date of Birth" column.It would seem to me that there should be an easy way to refresh the items (specifically the Excel Destination object that shows the columns from the underlying Excel template file) to where once the "Date of Birth" field has been added to the Excel template file, I can then easily open the Excel Destination object that maps to this underlying Excel template file and I suddenly see the presence of this newly-added "Date of Birth" field.What am I missing here? It's causing me a great slowdown to have to delete and rebuild each object involved within each of numerous data flow tasks just to be able to see/use/map a newly-added column from the Excel template file.Many thanks for any info. here.
↧
Connection String With Data Source = localhost
I was updating a package today and noticed that the ssis package has an SSIS variable holding the connection string for the OLEDB connection manager. The connection Mgr sets its connection string to the variables value in an expression. The DataSource for the connection string in the variable is hardcoded as "localhost". Am I assuming correctly that this will point to the databases on the server that the package is deployed to? The connection is used to point to the SSIS_Configurations table where the rest of the package configurations are held.I would think that the best practice would be to create an enviroment variable on each machine (Dev, QA, UAT, and Prod) and set all connection Mgrs to use the enviroment variable as its connection string with an enviroment variable config. What do you guys think?
↧
↧
(SSIS 2012) - XML Config Files - Dev vs Production Environment Question
I am developing a SSIS 2012 package that basically works with a bunch of incoming Excel spreadsheets and does ETL on them to get the dates into a more highly desired Excel schema via an Excel template file. I understand that I can go the route of using an XML configuration file to allow numerous variables to have their values changed via easy edits of the XML configuration file once moved to the production location where the package will run. The a bit confused over how to use an INDIRECT XML configuration option to allow the path and name of the configuration file to be dynamically recognized and work appropriately in both my development environment and the production environment. What I think I understand is that if you don't want to have to do additional work to have the different environments know dynamically where the XML configuration files are located via creation and usage of an Environmental Variable, the developer can simply cost to put the XML configuration file in the same location where all of the SSIS package files are located and that the process will find the XML configuration File in that location. Otherwise, if you want the XML config file to reside in a completely different location, you must create both a variable within SSIS as well as an Environmental Variable and somehow utilize both of those variables. Anything that can be communicated to me to make this a little clearer to me is much appreciated. I think I'm on the general right track with my understanding but I know I may be off a little. Examples and/or useful links any of you have found to related articles/tutorials that have helped you with this are appreciated, too, if you have any to share. Thanks in advance!
↧
HOW TO EXPORT A TABLE TO A CSV FILE --IN MS SQL SERVER 2008
[b][font="Comic Sans MS"]HiIam working on sql server and performed some queries and got required tables as results. Now I want to Export the table to a CSV file USING SCRIPT (only).help me if anyone knows.:w00t:[/font][/b]
↧
(SSIS 2012) - Directory.GetFiles --- How to Not Show Numeric Count Beside Each File
I'm using SSIS 2012 to create a comprehensive ETL package for a client to do a bunch of processing of incoming Excel files from clients of theirs.One part of the SSIS package is that I run the following to get a list of all files found within numerous subfolders to show in an email to the client:dim sFilesFound() as string = Directory.GetFiles("c:\ExcelFilesFound\","*.*",SearchOption.AllDirectories)The list of files returned is great, as it returns the full paths in all subfolders along with the file names. It does, however, include a running count of the files and this is something I really don't want if I can get around it. This is from memory, but it's something like this as far as what sFilesFound ends up containing:(01): "c:\ExcelFilesFound\Folder1\Test.txt"(02): "c:\ExcelFilesFound\Folder1\AnotherText.txt"(03): "c:\ExcelFilesFound\Folder2\Hello.txt"etc...I'd really just prefer to see:"c:\ExcelFilesFound\Folder1\Test.txt""c:\ExcelFilesFound\Folder1\AnotherText.txt""c:\ExcelFilesFound\Folder2\Hello.txt"etc...... instead of the numeric counts also shown at the beginning of each line showing each file path/name.Any suggestions here?Many thanks in advance for any tips any of you may have.
↧