I have a package that has an OLE DB source that queries a table and returns about 1.6 million rows, after which there is some processing on these records and they are then inserted into a staging table. The logic in the package passes ALL source records into the target OLE DB destination. After the package runs, I query the target table for a specific record, and the record is missing but the record was in the source table.So, I debug the package by adding a WHERE clause in the OLE DB source to constrain just to the record in question, run the package and the package passes 1 row through all the logic to the OLE DB destination. After the package runs to completion, I query the target table for the specific record, and the record IS in the target table.My source is SQL Server 2005, and my target is SQL Server 2005 (same database, different table). The OLE DB source and destination are using the same connection manager which is configured using the OLE DB\SQL Native Client provider.OK SSIS gurus, any idea what is going on here?
↧
The case of the missing records (SSIS Fast Load losing records?)
↧
Data Import from CSV file
I have a CSV file which is having record like mentioned below.[code="other"]UserName User Id TypeRamKumar ram MSADRamaC rama MSADRamesK rames MSADRameshv ramesh MSADRamKumar ramp PCGRamig rami PCGRamem rame PCGRampD ramp PCGRampD rampX MSAD[/code]Here I need to import this in a DB table for which DDL is like mentioned below[code="sql"]CREATE TABLE #USERTABLE([USERNAME] [varchar](100),[MSAD] [varchar](20),[PCG] [varchar](20))[/code]I need to import the data into the table.The logic is If Type is MSAD then it should update MSAD column of table with userid and if type is PCG then it should update PCG column also there should not be duplicate entry for a uername i.e. for RamKumar ther should be only one entry. with both MSAD and PCG column values.Expected output is mentioned below[code="sql"]UserName MSAD PCGRamKumar ram rampRamaC rama RamesK rames Rameshv ramesh Ramig ramiRamem rameRampD rampX ramp[/code]
↧
↧
high memory usage
hello experts,please go through the below configuration:-sql server 2005 sp2 windows server 2003 R2 (64 bit)CPU 2.13 GHzRAM 128 GBmax server memory 100GBmin server memory 40GBToday morning got an issue server was too slow and concernd team taken a screen shot of memory utilization till 102 GB, first concern, is any possibility that sql server can take more memory than what we have defined in max server memory??
↧
Conversion of DTS 2000 Packages to SSIS Pcakages in SQL 2008
Hi ,I have 65 DTS:2000 packages in our project. I just want to convert all these packages to SSIS packages using SQL Server 2008. Can anybody please help me out with easiest way of doing this large amount of packages ?Please also give me the steps of doing this as I'm doing it for the first time ...Thanks in advance !!!
↧
SSIS 2005 Import Excel (xlsx) file
Hi,I have xlsx file that I need to load it into SQL server 2005 table. When I create a Excel connection source (in SSIS 2005) and select my xlsx file, I get 'External Table is not in Expected Format' message. Any idea why this is happening? Any help would be appreciated.Thanks!
↧
↧
Creating Header AND Footer to a flat file destination file
Hi Guys,I am working on a financial project; the requirement is interface flat file should not contain any duplicated arrangement. (e.g. Check Sum) this requires the source extraction process to include quality checks to ensure that same arrangement is not included in interface files more than once.[b]Header Record [/b][b]Fields Type Value [/b]Record TypeCode String 000Record Format Version No Number 1Creation Date/Time Stamp DateTime 20060601013120Region String US [b]Trailer Records[/b]Record Type Code String 999Record Format Version No Number 1Creation Date/Time Stamp DateTime 20060601013120Region String USDetailRecordCount Number 35001Control Total Number -2993278934.87 [b]Every interface file will consist of the following record types:[/b]· Header record – The first record will be the header record.The purpose of the header is to provide identification information of the file that can be checked by the receving system for assurance that the correct file received. There is only one such record per file and it is the 1st record in the file. · Detail records – There will be zero or many detail records. They will follow the header record.· Trailer record – The last record will be the trailer record.The purpose of the trailer record is to provide identification information of the file and control data that can be checked by the receiving system for assurance that the correct file received and provide a set of batch control total that can be used by the receiving system to check if the content of the file has not been altered since the file was created. [b]Control Totals:[/b] Detail Record count, excluding the header record and trailerrecord.Control Total 1 = Simple sum of “principle_amount_1” of every detail arrangement. There is only one such record per file and it is the last record in the file. I need some help here, how to approach this requirement. Do I need to create separate tables for header and footer with value types. How to check sum header and footer as per above requirement ? Any article or code related to above requirements please let me know I will try my best to figure it out.Management insist me to do it MS SSIS. need some help. Thanks in Advance, D
↧
Set Excel Connection Manager connection string dynamically using variables
I have developed an SSIS package (using BIDS 2005) that requires almost everything to be set dynamically; Connections Managers, package variables, SQL statements executed by Execute SQL tasks etc.The reason for this is that the package is designed to be a generic utility package that produces Excel files in various directories and is called from a stored procedure. The stored procedure sets some of the package variables; others are set using an xml variable. It is based on [url=http://beyondrelational.com/modules/2/blogs/66/posts/9928/calling-a-ssis-package-from-a-stored-procedure.aspx]this[/url] excellent article by Jeff Wharton.The Excel Connection Manager is set using a package variable and that variable in turn is constructed from a number of other variables. Everything works fine when those other variables are hard-coded, but if one of them is itself set dynamically, the package fails with an error which basically says that the Excel Connection Manager cannot be acquired because it is incorrectly formed. I have set the [b]'Delay Validation'[/b] property on the Excel Connection Manager to True, but this doesn't seem to make a difference.To better illustrate the situation, my Excel Connection Manager's connection string is set, via an Expression to use a variable, [b]User::Conn_ExcelFile[/b]The User::Conn_ExcelFile variable has its [b]EvaluateAsExpression[/b] property set to True and the expression it uses is: [code="plain"]@[User::vDriver] + @[User::vExtendedProperties] + @[User::vExcelFile][/code]The [b]User::vDriver[/b] and [b]User::vExtendedProperties[/b] variables are hard-coded, but the [b]User::vExcelFile[/b] variable is set dynamically using a /SET option in the dtexec command line. I can see using breakpoints that the User::vExcelFile variable is indeed being correctly set to the value specified by the /SET option, but the value is blank when the User::Conn_ExcelFile variable is set.[b]Q:[/b] Are variables set dynamically in the order in which they are created (in which case I can resolve this easily by ensuring that my User::Conn_ExcelFile appears further down the list of variables than my User::vExcelFile variable)? [b]Q:[/b]Is it even possible to create this chain of dynamically-set variables?ThanksLempster
↧
How to schedule ssis package to run only on Business working days ?
hi friends, I have a stored procedure (performing some transformations) , which is in SSIS package.I need to schedule SSIS package to run only on Business working days.How can I implement this?can anyone tell me please ? Thanks,Komal
↧
Import .dbf files in sql server 2005
Hi there, I have a .dbf file which i want to import in sql server 2005. When I right click on the database and click Import, I don't know what Data Source I should be using. Is it even possible to import .dbf files?Thanks
↧
↧
DATA TASKS FAILS...IN SSIS
i have a dataflow; loading from flat file to sql table. There are truncation warnings, but actual data will not be trunacted. However, i used to have same issues, but when i used to change my seetings " on truncation -ignore" it used to work. tHis time this seetings is not helping. However when i change to " on error-ignore" it works. below is the details error message that i get.---------------[OLE DB Destination [42]] Error: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005. An OLE DB record is available. Source: "Microsoft OLE DB Provider for SQL Server" Hresult: 0x80004005 Description: "Invalid character value for cast specification.". An OLE DB record is available. Source: "Microsoft OLE DB Provider for SQL Server" Hresult: 0x80004005 Description: "Invalid character value for cast specification.". [OLE DB Destination [42]] Error: There was an error with input column "Col006" (253) on input "OLE DB Destination Input" (55). The column status returned was: "The value could not be converted because of a potential loss of data.". [OLE DB Destination [42]] Error: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR. The "input "OLE DB Destination Input" (55)" failed because error code 0xC0209077 occurred, and the error row disposition on "input "OLE DB Destination Input" (55)" specifies failure on error. An error occurred on the specified object of the specified component. There may be error messages posted before this with more information about the failure. [DTS.Pipeline] Error: SSIS Error Code DTS_E_PROCESSINPUTFAILED. The ProcessInput method on component "OLE DB Destination" (42) failed with error code 0xC0209029. The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running. There may be error messages posted before this with more information about the failure. [DTS.Pipeline] Error: SSIS Error Code DTS_E_THREADFAILED. Thread "WorkThread0" has exited with error code 0xC0209029. There may be error messages posted before this with more information on why the thread has exited.
↧
Load data from excel , is it possible?
col1 col2 col35.0 6.0 3.59.5 9.8 6.3 9.0 6.0 8.0Col4 col5 col69.5 6.0 8.0 6.3 9.3 3.98.7 8.0 8.89.0 8.3 6.98.6 9.0 6.2I have data like this in excel, how to load into 2 different tablesfirst table contain col1,col2,col3 as columnssecond table contain Col4,col5,col6 as columns.Note: Here the number of columns are fixed(3).
↧
Help with Multi-level XML
I am teaching myself SSIS 2005.I created an XML fileand a table in SQL Server.the XML looks like<Sightings> <Bird> <category>Shorebird</category> <Name>Greater Yellowlegs</Name> <Site> <Location>Winthrop</Location> <Date> 10/15/2012</Date> </Site></Bird></Sightings>The site element has sub elements and I am not sure how to map the fields to the database table fields
↧
for each loop files in folder starting with @[User::strFileNameStart] + "*.csv"
I have an folder with multiple files . For passing the folder, i have set the Directory expression as @[User::strFinSourceFolder] + "\\"I want to select files starting with USA. File loop works fine when i add the Files textbox as USA*.csv I add the parameter instead of USA in the files textbox as@[User::strFileNameStart] + "*.csv"But things are not even entering into the loop ...Please help me how to set the filename start in the for each loop
↧
↧
SSIS Package step FTP
I have a step that ftp out to a location to grab a file. each month the file name changes, is there a way to filter or have the ftp step pick up the file.Currently this is the file name 56730_LAF_Layout.xlsx. The only part the changes each month is the first 5 numbers. I tried *_LAF_Layout.xlsx but didnt work does anyone have a suggestions on this?Thanks in advance
↧
Pass SQL Execute task output to target tbl(Data flow).
Hi,we need to insert random values to target table.We are using the query in SQL Execute task with single row result set & in the Result Set tab set with user variable(intRdmid).declare @PriID intselect top 1 @PriID = code_id from Pri_codes_master where code_desc like '%assignment%'ORDER BY NEWID()select @PriIDBut getting error as...[Execute SQL Task] Error: An error occurred while assigning a value to variable "intRdmid": "Unable to find column @PriID in the result set.". I want to use this value in data flow task to insert in target table.How can I implement this?can anyone tell me please ? Thanks,
↧
DATA CONVERSATIONS TASKS FAILS IN SSIS
Hi, The problem was long enough to take my whole day. To make story short-- i have flat file with uneven number of columns in each row. so i have a script component to handle that part. But now the problem is I added a data conversation in my dataflow which is converting string [DT_str] 50 into database timestamp, which is my sql table destination datatype. Following is the error i get when i run the task--ERROR:--[[i]Data Conversion [2628]] Error: Data conversion failed while converting column "Col3" (923) to column "Copy of Col3" (2651). The conversion returned status value 2 and status text "The value could not be converted because of a potential loss of data.". [Data Conversion [2628]] Error: Data conversion failed while converting column "Col3" (923) to column "Copy of Col3" (2651). The conversion returned status value 2 and status text "The value could not be converted because of a potential loss of data.". [/i]the follow of my datatask is i have flat file and i am inporting all columns(i.e. my row of different varying lengths) into single column.then i have a script components to handle the uneven length of columns based on vertical line.then i have a data conversation task that converts col3 string into timestampand finally i load data into sql destination -----When i change the option on error , ignore the failure, it works , all the data gets loaded.however. my team wants me to find alternatives. CAn someone help me other way i can achive the same results or am i doing something wrong on my conversation? thanks, kumar
↧
SSIS LOOKUP QUESTION
Wanted to use lookup transformation for the following:- Pull records from oledb source- Lookup in oledb destination first to see if record exists- if record does not exist, send to destination as insert.The result just dumps everything into the destination when the Destination is Empty(In my case it is a blank table in the starting)I have tried no cache option also but thats not working too...as there are duplicates...[b]I don't want any duplicates in my loookup column....[/b]PLEASE HELP...THIS IS KILLING ME
↧
↧
Help to compare two tables in SSIS
I want to insert values in to “Table B” from “Table A”. Both the tables has only two columns and same.Table B is destination table and it's constant. Table A is getting Updated every week. So I have to extraxt the data from table A and compare with table B and updated.If the data is not exists in “Table B” I want to insert values from “Table A”If the data is exists in both tables do not do anything. (Like eliminating the duplicates).Insert only updated data from Table A in to table BI want to implement this by SSIS. Which component can i use in SSIS , and how ?Tell me in step by step, I am very new in SSIS
↧
how to create ssis package to copy table from one server to another server
how to create ssis package to copy table from one server to another server using BIDS
↧
Data conversion in SSIS
Hi everyone,I am newbie to SSIS, just picked that up ytd. Currently i am given a sample view table to work with. However the EVENT_D and EVENT_T field are in text data type in my access 2003. Its a text field in DD/MM/YYYY HH:MM:SS format respectively. Can anyone guide me how to convert this two field into date and time format respectively in SSIS?The second qns is how can i concat this two field into one derived column EVENT_DT as datetime in DD?MM?YYYY HH:MM:SS format?Any help are greatly appreciated.Thanks
↧
